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Preface 



Over the past years enormous progress has heen made in the microsystems 
area in transforming research results into marketahle products. Nearly all eco- 
nomic sectors did benefit from these developments. Automobiles today are 
inconceivable without microsystems. New and improved functions in the car 
related to safety, performance, comfort and emission reduction - being in 
many cases the unique selling propositions of a new automobile product - are 
in the majority of cases based on microsystems. ABS, break assistance, stabil- 
ity control and further safety features, applications in air condition systems, 
powertrain and engine management are only some examples for the indis- 
pensable role of microsystems in automobiles. Microsystems and their under- 
lying technologies are in many cases even the driving forces in order to satis- 
fy new customers' requirements. 

Even more than in the past microsystems are the keys in order to face the 
enormous challenges to be faced by the automotive industry in the future: 

Vehicles have to satisfy higher standards concerning environmental issues: 
requirements concerning reduced air pollution, energy consumption and sus- 
tainability requirements will have to be met by incremental and radical inno- 
vations related to powertrain, fuel and material. 

"TYaffic, in particular automobile traffic, threatens to become a victim of its 
own success", says a car manufacturer. In fact, the negative side of an 
improved mobility consists of severe consequences for the environment and 
the economy due to congestion problems. Technological innovations and effi- 
cient organisational systems will help to solve current and future problems. 

In spite of remarkable achievements in vehicle and road safety over the last 
decades there still is a long way to go towards the future vision of an accident- 
free traffic. Active safety systems for collision avoidance represent therefore 
the most efficient safety systems for the future. Better driver information and 
assistance on the driving conditions are part of the concept. 

In all three of the mentioned areas - environment, mobility and safety - 
microsystems are playing a key role. More than in the past, however, integra- 
tion aspects are becoming of increasing importance: 

Microsystems have to be seen as part of an integral concept of a communicat- 
ing and acting system embedded in new architectures and in structures of car- 
to-car and/or car-to-road communication. 




This book published in occasion of the International Forum of Advanced 
Microsystems for Automotive Applications 2004 reflects the state-of-the-art of 
automotive R&D particularly in the area of automotive safety. By that it follows 
the priorities set by the European automotive industry and public authorities 
as for example the European Commission or the German Federal Ministry for 
Education and Research. 

In the safety context a major role will play this year vision systems, emer- 
gency breaking and tire pressure measurement. Given the importance of inte- 
gration aspects the networked vehicle issue will not be missing. 

My explicit thanks go to the authors for their valuable contributions to this 
publication and to the members of the Honorary and Steering Committee for 
their commitment and support. Particular thanks are addressed to Gloria 
Pellischeck, CLEPA, and Berthold Ulmer, Daimler Chrysler for the inspiring 
input we received. 

I would like to thank the European Commission for their financial support 
through the Innovation Relay Centre Northern Germany as well as the gov- 
ernmental authorities in Berlin and Brandenburg. Thanks go also to the sup- 
porting organisations Investitionsbank Berlin, mstnews and Victorian TTade 
Commission, Australia. 

Last but not least, I like to express my sincere thanks to the Innovation Relay 
Centre team at VDI/VDE-IT, particularly to Jasmin Mehrgan and 
Angelika Boskugel for preparing this book for publication, and Jurgen Valldorf, 
as the AMAA project manager. 

Teltow/Berlin, March 2004 



Wolfgang Gessner 




Public Financers 



Berlin Senate for Economics and Technology 

European Commision 

Ministry for Economics Brandenburg 

Supporting Organisations 

Investitionsbank Berlin (IBB) 

State Government of Victoria, Australia 
mstnews 

Co-Organisators 

European Council for Automotive R&D (EUCAR) 
European Association of Automotive Suppliers (CLEPA) 
Advanced driver assistance systems in Europe (ADASE) 




Honorary Commitee 



Domenico Bordone 



Gunter Hertel 



Remi Kaiser 



Gian C. Michellone 



Karl-Thomas Neumann 



President and CEO 
Magneti Marelli S.p.A., Italy 

Vice President Research and Technology 
DaimlerChrysler AG, Germany 

Director Technology and Quality 
Delphi Automotive Systems Europe, France 

President and CEO 
Centro Ricerche FIAT, Italy 

Head of Electronic Strategy 
VW AG, Germany 




steering Commitee 



Dr. Giancarlo Alessandretti 
Alexander Bodensohn 

Wilhelm Bois 
Serge Boverie 
Roger Grace 

Henrik Jakobsen 
Horst Kornemann 

Hannu Laatikainen 
Dr. Peter Liden 
Dr. Torsten Mehlhorn 
Dr. Roland Miiller-Fiedler 
Paul Mulvanny 
Dr. Andy Noble 

Gloria Pellischek 
David B. Rich 

Dr. Detlef E. Ricken 

Jean-Paul Rouet 
Christian Rousseau 
Ernst Schmidt 
John P. Schuster 
Bob Sulouff 
Dr. Gerd Teepe 
Berthold Ulmer 
Dr. Jurgen Valldorf 
Egon Vetter 

Dr. Matthias Werner 
Arnold van Zyl 



Centro Ricerche FIAT, Orbassano, Italy 
DaimlerChrysler AG, 

Frankfurt am Main, Germany 
Audi AG, Ingolstadt, Germany 
Siemens VDO Automotive, Toulouse, France 
Roger Grace Associates, 

San Francisco, USA 

SensoNor A.S., Horten, Norway 

Continental Automotive Systems, 

Frankfurt am Main, Germany 
VTI Technologies Oy, Vantaa, Finland 
AB Volvo., Gdteborg, Sweden 
Investitionsbank Berlin, Berlin, Germany 
Robert Bosch GmbH, Stuttgart, Germany 
QinetiQ Ltd., Farnborough, UK 
Ricardo Consulting Engineers Ltd., 
Shoreham-by-Sea, UK 
Clepa, Brussels, Belgium 
Delphi Delco Electronics Systems, 

Kokomo, USA 

Delphi Delco Electronics Europe GmbH, 
Riisselsheim, Germany 
Johnson Controls, Pontoise, France 
Renault S.A., Guyancourt, France 
BMW AG, Munich, Germany 
Motorola Inc., Northbrook Illinois, USA 
Analog Devices Inc., Cambridge, USA 
Motorola GmbH, Munich, Germany 
DaimlerChrysler AG, Brussels, Belgium 
VDIAfDE-IT, Teltow, Germany 
Ceramet Technologies, 

Melbourne, Australia 

Deutsche Bank AG, Berlin, Germany 

EUCAR, Brussels, Belgium 



Conference chair; 

Wolfgang Gessner VDIAfDE-IT, Teltow, Germany 




Table of Contents 



Introduction 

Bridging the Gap between Technoiogy Deveiopment and Market 
Impiementation for Vehicie Safety Systems 3 

F. Minarini, European Commission 



Safety 

Monoiithic Acceierometer for 3D Measurements 1 1 

T. Lehtonen, J. Thurau, VTI Technologies Oy 

Highiy Integrated Tire Pressure Monitoring Soiutions 23 

M. Osajda, Motorola 

Low-Cost, Singie-Chip Ampiified Pressure Sensor in a Mouided 
Package for Tire Pressure Measurement and Motor Management 39 

R. Bornefeld, W. Schreiber-Prillwitz, O. Stover, Elmos Semiconductor AG; 

H. V. Allen, M. L. Dunbar, J. G. Markle, Silicon Microstructures, Inc.; 

I. van Dommelen, A. Nebeling, J. Raben, Eurasem B.V. 

Roiiover Detection Systems 51 

J. Haun, Delphi Electronics and Safety 

Low-Cost Infrared Imaging Sensors for Automotive Appiications 63 

M. Hirota, Y. Nakajima, M. Saito, M. Uchiyama, Nissan Motor Co., Ltd. 

Appiication of Micro Scanning Laser Doppier Vibrometer for 
Vibrationai Veiocity and Resonance Frequency Measurement 
on Micro Eiectro Mechanicai Devices (MEMS) 85 

P. Castellini, B. Marchetti, E. P. Tomasini, Universita Politecnica delle Marche 

Mobiie Vision - Deveioping and Testing of Visuai Sensors for 

Driver Assistance Systems 95 

R. Preib, C. Gruner, T. Schilling, H. Winter, Aglaia Gesellschaft fur 
Bildverarbeitung und Kommunikation mbH 




Silicon Capacitive Absolute Pressure Sensor Elements for 
Battery-less and Low Power Tire Pressure Monitoring 

J. Thurau, J. Ruohio, VTI Technologies Oy 

Automotive Applications for the ALASCA Laser Scanner 

B. Nitsche, Hella KG Hueck & Co.; R. Schulz, IBEO AS GmbH 

Sensor Fusion for Multiple Automotive Active Safety and Comfort 
Applications 

N. Kaempchen, K. C. Fuerstenberg, A. G. Skibicki, K. C. J. Dietmayer, 
University of Ulm 

Dealing with Uncertainty in Automotive Collision Avoidance 

J. Jansson, Volvo Car Corporation 

Paroto Project: Infrared and Radar Data Fusion for Obstacle 
Avoidance 

Y. Le Guilloux, J. Lonnoy, R. Moreira, SAGEM SA; C. Blanc, J. Gallice, 

L. Trassoudaine, LASMEA; H. Tattegrain, M.-R Bruyas, A. Chapon, INRETS 

From Passive to Active Safety Systems 

T. Gornig, Conti Temic microelectronic GmbH 

A Low-g 3 Axis Accelerometer for Emerging Automotive 
Applications 

B. Grieco, D. Ausilio, F. Banfi, E. Chiesa, G. Frezza, E. Lasalandra, 

P. Galletta, M. Garavaglia, A. Mancaniello, A. Merassi, F. Pasolini, G. Sironi, 

F. Speroni, M. Tronconi, T. Ungaretti, B. Vigna, S. Zerbini, 

ST Microelectronics 



Powertrain 

Autonomous Sensor Systems for Car Applications 

A. Bodensohn, R. Falsett, M. Haueis, M. Pulvermuller, DaimlerChrysler AG 

European on Board Diagnostic Catalyst Efficiency Measured 
on Board and on Test Bench 

A. Celasco, F. Cavallino, Powertrain Italy 



109 

119 

137 

165 

181 

201 

211 

225 

233 




Comfort and HMI 



Multifunctional Sensor to Detect Environmental Parameters 247 

N. Pallaro, F. Visintainer, M. Darin, E. Mosca, Centro Ricerche Fiat 

Windshield Fogging Prevention by Means of Mean Radiant 
Temperature Sensor 267 

S. Mola, G. Lo Presti, M. Magini, N. Presutti, C. Malvicino, Centro Ricerche 
FIAT; G. Bisceglia, M. Mandrile, A. Tarzia, G. Caviasso, Fiat Auto 



Networked Vehicle 

Sensor Integration in the Mechatronic Environment 281 

G. Teepe, Motorola 

Networking In-Vehicle Entertainment Devices with HAVi 295 

A. Leonhardi, T. Gschwandtner, M. Simons, DaimlerChrysler AG; 

V. Vollmer, G. de Boer, Robert Bosch GmbH 

Digital Audio Broadcasting - Solutions for the Automotive 
Environment 313 

S. Evans, Frontier Silicon 

Automatic Functional System Test of Complex Automotive Devices 325 

A. Guerra, IXFIN Magneti Marelli Sistemi Elettronici 

Telematics Digital Convergence - How to Cope with Emerging 
Standards and Protocols 335 

K. Parnell, Xilinx 



Appendices 

Appendix A: List of Contributors 351 



Appendix B: List of Keywords 



357 




Introduction 




Bridging the Gap Between Technology Development and 
Market Implementation for Vehicle Safety Systems 



F. Minarini, European Commission 



Abstract 

With its 375 million road users, mobility and transport is a concern 
for citizens throughout Europe. Modern society depends on mobility 
as an efficient transport system is the engine of our economy. The 
transport sector employs more than 10 million people with expendi- 
ture of more than 10% of the Gross Domestic Product (GDP) in 
Europe. The automotive sector manufactures about 17 million vehi- 
cles per year and employs with its suppliers close to 2 million people 
in Europe, with a turnover of 452 billion € world-wide [1]. The auto- 
motive telematic market, comprising of sales of platforms and serv- 
ices is experiencing rapid growth in market penetration, and accord- 
ing to some market studies, it will reach an annual revenue as high 
as 8,5 billion € in Europe in 2007, up from 1 billion € in 2000. 

In the year 2001 there have been 1.300.000 accidents in Europe that 
caused 40.000 fatalities and 1.700.000 injuries, at an estimated cost 
of 1 60 billion €, or 2 % of the GDP. On a personal level, these accident 
figures translate to one third of us being injured in an accident at 
some point of our lives. The psychological damage to the victims and 
their families cannot even be estimated. 

Thanks largely to the efforts of the industry, vehicles today are 
inherently safer, cleaner and more recyclable than before. Thanks to 
improvements in the crash-worthiness of the vehicles, safety belts, 
ABS and other inventions, the vehicles are now four times safer than 
in 1970; this has largely contributed to the reducing by 50% the 
number of deaths in EU15 from 1970, while the traffic volumes have 
tripled during the same period. But the societal costs of road trans- 
port are still far too high, and new goals needed to be set. This is 
why, in its White Paper "European TTansport Policy for 2010: Time 
to Decide", for the first time the European Commission has set an 
ambitious target to reduce by half the number of road fatalities by 
2010 [ 2 ]. 
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1 From Passive Safety to Active Safety and Accident Prevention 

During the last decade, the European Union, Member States and the automo- 
tive industry have been actively involved in improving road safety through 
both accident prevention and injury reduction. Most of the accident preven- 
tion measures have focused on the driver, while the measures to reduce the 
consequences of an accident have primarily focused on the vehicle, through 
improved passive safety such as crashworthiness, seatbelts airbags and con- 
ventional active safety systems such as braking and lighting. Passive Safety 
measures have been proven to be a very effective method for the reduction of 
the car accident trauma. 

These combined actions have contributed to the continuous reduction of the 
number of fatalities on European roads. The number of deaths in EU15 has 
halved from 1970, while the traffic volumes have tripled during the same peri- 
od [3]. Nevertheless, the number of road accidents and the number of road vic- 
tims are still unacceptably high in the European Union. Furthermore, the con- 
tribution of many of these "conventional" safety measures is reaching its lim- 
its, and further improvements in safety by these measures are becoming more 
and more difficult to achieve at a reasonable cost. This is why the in-vehicle 
passive safety has to be complemented by introducing on the markets more 
advanced in-vehicle and co-operative active safety systems. 



2 Towards a New Generation of Intelligent Safety Systems Through 
Research 

During the last decade in Europe both the industry and the public sector have 
invested heavily in Research and Development (RTD) in the use of Information 
and Communications Technologies (ICT) in in-vehicle safety technologies and 
subsystems. While the bulk of the work has been done by the industry, the 
EU's Research Programmes have contributed in realising leading edge tech- 
nologies, systems and applications. 

The EU has played a role in this research since the DRIVE Programme in 1988. 
Under the EU's Fourth Framework Programme for Research, Technological 
Development and Demonstration (1994-1998), the Telematic Applications 
Programme, helped in realising leading-edge systems and applications. The 
Information Society Technologies (1ST) programme [4] continues research in 
technologies and applications systems aiming at safer, cleaner and more effi- 
cient transport, with specific focus on intelligent safety and Advanced Driver 
Assistance Systems and supporting technologies. The Intelligent Vehicle clus- 
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ter of the 1ST Programme has over 40 projects, with total budget of over 
1 50 million € and Community contribution of over 80 million €. Research into 
safer vehicles and infrastructures as well as accident databases and human 
behaviour in road transport is also undertaken in the Competitive and 
Sustainable Growth programme (5**^ Framework Programme) [5], 

The development of appropriate sensors, actuators and processors, has already 
permitted wide spread implementation of systems, which help the driver to 
maintain control of the vehicle even when he has exceeded its "normal" limits 
of handling. Examples of such systems are Anti-lock Braking Systems (ABS) 
and Electronic Stability Programme (ESP). The latter one for example detects 
the onset of a slide and automatically applies the brakes to individual wheels 
to correct the slide and prevent spinning. ESP is now optional or even standard 
on many current passenger cars. Statistics from one car manufacturer shows 
that in 2001 there was a 4% reduction of accidents, compared to the year 
before, that could be directly attributed to ESP. The reduction of roll-over acci- 
dents thanks to ESP have also been estimated to be 12%. 

The further development of intelligent active safety systems will require sub- 
stantial RTD efforts. The current Sixth Framework Programme for Research 
and Technological Development (2002-2006) is offering new funding opportu- 
nities for RTD in intelligent integrated safety systems including accidentology 
and Advanced Driver Assistance Systems and technologies. The new instru- 
ment, Integrated Project, which is designed to generate the knowledge 
required to address major societal challenges, holds the potential to especially 
suitable for research in this area. Integrated Projects are intended to build a 
critical mass of activities and resources needed for achieving ambitious, clear- 
ly defined scientific and technological objectives. In the area of integrated 
safety systems, for example, an Integrated Project called PREVENT will start 
in February 2004. This project regroups more than 50 partners for a total 
budget of almost 80 million € with the European Commission contributing 
with 30 million € for a duration of 4 years. 



3 Intelligent Vehicle Safety Systems 

We know that almost 95% of the accidents are at least partly due to the human 
factor. In almost three-quarters of the cases the human behaviour is solely to 
blame. This apparent mismatch between driver skills and situation complexi- 
ty can be addressed by improvements in three factors: the driver (education 
and training); the environment (intelligent infrastructure) and the vehicle (in- 
vehicle safety systems). 
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Intelligent Vehicle Safety Systems use Information and Communications 
Technologies for providing solutions for improving road safety in particular in 
the pre-crash phase when the accident can still he avoided or at least its sever- 
ity significantly reduced. With these systems, which can operate either 
autonomously on-hoard the vehicle, or be based on vehicle-to-vehicle or vehi- 
cle-to-infrastructure communication (co-operative systems), the number of 
accidents and their severity can be reduced, leading egually to a reduction of 
the number of fatalities and injuries. 

Collisions during lane changes and involuntary lane departure, for example are 
two of the most important causes of accidents. This problem requires in-vehi- 
cle technology to help detect and warn drivers of vehicles in adjacent lanes or 
when the vehicle is about to unintentionally depart from the lane. According 
to the National Highway Traffic Safety Agency (NHTSA) in the US, lane change 
and merge collisions could be cut in half by new technologies. In Europe, a 
Dutch study expects a reduction of 37% in all side impact collisions and a 
reduction of 24% of the single vehicle accidents due to lane departure crash 
avoidance. 

Location-enhanced emergency calls like in-vehicle e-Call have their primary 
benefit to society of saving lives and in offering an increased sense of securi- 
ty. This is achieved by improved call routing obtaining faster and improved 
information for dispatching relevant resources, and most importantly 
improved information to locate the caller. What is of paramount importance 
here is that the relevant resources are delivered to the person in need as soon 
as possible, this can save up to 10% of the fatalities. 



4 eSafety 

The potential contribution of the introduction of Intelligent Vehicle Safety 
Systems for enhancing road safety and security has already been demonstrat- 
ed by the industry in a number of European research and technological devel- 
opment (RTD) projects. However, to realise the potential benefits, the new sys- 
tems have to be widely deployed in the marketplace and the existing gap 
between systems development and the market implementation of these sys- 
tems has to be drastically reduced. Accelerating the development and deploy- 
ment Intelligent Vehicle Safety Systems is the main objective of the eSafety 
initiative that was established during the High Level Meeting on 25 April 
2002. The eSafety initiative is a public/private partnership that it has been cre- 
ated together by the European Commission and the industry mainly repre- 
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sented through ACEA and ERTICO. The actual work started in the High-level 
group when the e-Safety Working Group was officially established . 

In November 2002, the Working Group published its Final Report, which the 
2°'i eSafety High-Level meeting endorsed as the basis for further actions. 

This Final Report contains 28 recommendations for the European Commission, 
the Member States, road and safety authorities, automotive industry, service 
providers, user clubs, insurance industry and other stakeholders. The recom- 
mendations aim at improving road safety by integrated safety systems that 
use advanced ICT for providing new, intelligent solutions which address 
together the involvement of and interaction between the driver, the vehicle 
and the road environment. 

Acting upon these recommendations, the Commission organised the first ple- 
nary meeting of the eSafety Forum in Brussels on April 22, 2003. The Forum 
brought together 150 representatives of the European automotive and 
telecommunication industries, providers of intelligent transport systems, infra- 
structure operators, public authorities and the Commission. Following the lines 
set out by its four preparatory working groups, the forum adopted recommen- 
dations on how to implement eCall (in-vehicle emergency calls). Accident 
Causation Data, a better Human-Machine Interaction and a stimulating 
Business rationale, and set up four new Working Groups covering International 
Cooperation, Real-time Traffic and TTavel Information, Research and 
Development priorities and Road Maps. The second eSafety Forum took place 
in Madrid on November 17, 2003 on the occasion of the ITS (Intelligent 
Ifansport Systems and Services) World Congress. 

The Commission has also adopted a Communication on "Information and 
Communications Technologies for Safe and Intelligent Vehicles". This 
Communication lays down a set of measures for supporting the industry in 
developing safer and more intelligent vehicles and enabling their rapid mar- 
ket introduction. The proposed actions fall into three categories: (I) Promoting 
Intelligent Vehicle Safety Systems, (II) Adapting the Regulatory and 
Standardisation Provisions and (III) Removing the Societal and Business 
Obstacles. 

As the car park of vehicles with telematic grows, the market will shift towards 
services, further integrating the automotive market with two other key indus- 
trial sectors in Europe: Mobile Communications and Information Technology. 
As a whole. Information and Communications Technologies (ICT) holds the 
potential to play a key role in the convergence of these sectors towards the 
Intelligent Vehicles of the future that through the eSafety initiative and the 
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commitment of the Industry and the public bodies like for example the 
European Commission and the Member States, can contribute to the set target 
of reducing road fatalities by 50% by 2010. 
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Monolithic Accelerometer for 3D Measurements 



T. Lehtonen, J. Thurau, VTI Technologies Oy 



Abstract 

Bulk micromachining low-g accelerometers for single axis measure- 
ments are today's state of the art technology for measuring low-g 
range signals in automotive, medical and other applications - also 
referred as "human scale motion". The latest trend in silicon micro- 
machining is smoothening the border between bulk and surface 
micromachining. A combination of SOI (Silicon On Insulator) wafers 
and DRIB (Deep Reactive Ion Etching) as new technologies is lead- 
ing to smaller and more cost efficient micromachined inertial sen- 
sors. A novel capacitive 3-axis accelerometer element is the first 
example of this combination by VTI Technologies. The key task was 
to overcome the physical limitation to measure a 3-dimensional 
acceleration signal with a 2-dimensional silicon element that suits to 
modern packaging constraints. It will be shown how the new sensor 
element is working and that the similarity of the manufacturing 
process with today's mass produced VTI single axis accelerometer, 
will lead to a fast commercialization of the novel 3-axis design. 
Additionally the utilization of this technology platform for further 
projects will be sketched. 



1 Starting Position 

Automotive inertial sensor road maps show a six degrees-of-freedom inertial 
measuring unit, where a three-axis accelerometer is needed for measuring lin- 
ear motion. The three-axis accelerometer is also expected to open up new mar- 
kets for accelerometer based motion sensing, navigation and inclination meas- 
urement in portable electronic devices [1]. Several previous attempts to design 
and manufacture a three-axis accelerometer are reported in the literature 
[2-5]. VTI Technologies has been developing and producing single axis low-g 
bulk micromachined capacitive accelerometers since the early 90's and has 
achieved a leading position with over 10 million annual production [6, 7]. Great 
deal of recent research and development work has been done to develop a new 
concept for a single element multiple axis accelerometer. 
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The approach described in this paper is a 3-axis accelerometer which requires 
a high sensitivity system consisting of springs and a proof mass or several 
masses, whose position is observed capacitively and further mapped to accel- 
eration information by an electrical circuit. Several requirements for the sen- 
sor element are existing such as: mass movement, conversion to an electrical 
signal and the mapping algorithm should yield a high signal to noise ratio and 
a linear acceleration response with negligible cross axis sensitivity. In the ideal 
case each measured axis should be mechanically equal in sensitivity and fre- 
quency response - or even better, freely adjustable over a wide range. The 
next important specifications are temperature dependence and shock resist- 
ance. It isn't less important that the accelerometer should be manufacturable 
based on feasible low cost processes, have small size and should enable low 
cost interconnecting and packaging. 

VTl inertial sensor technology is based on bulk micromachining using six inch 
wafers to meet all above mentioned requirements. Bulk micromachining tech- 
nology enables the formation of large inertial masses which have high sensi- 
tivity, reliability and resistance to stiction effects. Five layer sensor structure 
enables to achieve large capacitances using sensor element area effectively. 
35% of the area is covered by the capacitor electrode. An effective wafer area 
usage results in small size and low cost. 



2 Operation Principle 

The structure is based on multiple proof masses suspended asymmetrically 
with torsion springs attached close to the surface of the silicon wafer. An iner- 
tial mass with relatively high thickness attached in this way is sensitive in ver- 
tical and horizontal directions of the wafer. The resultant sensing direction 
depends on the location of the proof mass center of gravity with respect to the 
suspension points. The horizontal surfaces of the mass and metal electrodes on 
the capping wafers on both sides of the mass form a differential capacitor sys- 
tem. When the mass is excited by acceleration force, it will rotate. The move- 
ment changes the electrode distances and thus the capacitances. Figure 1 illus- 
trates a single mass operation and fixed electrodes. Solid line in figure lb indi- 
cates the mass position in the rest and dashed line mass position when excit- 
ed in sensing direction. 
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A minimum of three spring-proof mass systems is needed for a 3-axis 
accelerometer. By measuring the positions of the proof masses capacitively, 
the three orthogonal components of the acceleration vector can be calculated 
as linear combinations of the position signals from several proof mass-elec- 
trode systems. 





Fig. 1. Single mass function principle (left), as cross section (right). 

VTl's 3-axis capacitive accelerometer consists of four triangular spring-proof 
mass systems located at 90° angles from each other. Masses form totally eight 
capacitors and four differential capacitor pairs with fixed electrodes. Figure 2 
shows the spring-proof mass systems in a Cartesian co-ordinate system as well 
as a photo of the spring system made from one of the prototypes. 




Fig. 2. Complete spring-proof mass systems with spring detail. 

When acceleration load is applied on sensor, the masses will rotate according- 
ly. Figure 3 a shows an example of mass response when acceleration load is 
purely in positive x-direction. Acceleration makes masses 1 and 2 rotating 
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clockwise resulting in decreasing electrode gap for electrode 2 and 3. 
Correspondingly electrode gap for electrodes 1 and 4 will increase. Masses 3 
and 4 are not affected and capacitance change is not recognised. Figure 3b 
shows the situation when acceleration in positive z-axis direction is applied. As 
the result, masses 1 and 3 will rotate counter clockwise and masses 2 and 4 
clockwise. The acceleration resultant can be calculated from mass signals. 
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Fig. 3. Mass modes when excited in a) x-direction and b) z-direction. 




Fig. 4. 



Projected resulting force vectors. 
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The concept of four inertial masses enables very similar linear responses for 
all three measurement axes. Measurement for X- and Y-axis can be adjusted 
independently for different mechanical sensitivities and frequency bands. Z- 
axis measurement properties are dependent on X- and Y-axes properties, but 
still have equally high performance. 

Figure 4 shows the same system again but now with projection in two planes. 
A set of each two of four resulting force direction vectors is perpendicular to 
each other orientated 45° to the plane. The 45° angle out of the plane was cho- 
sen as the best result after various optimisations within the given degree of 
freedom. 



3 Sensor Element 

For best system integration it is necessary to realise a very compact sensor ele- 
ment. The inertial mass-spring systems are suspended to the frame of the sen- 
sor, which encloses each mass to its own cavity according to figure 2. These 
four cavities add up to a square of 2.65 mm by 2.65 mm. The structural wafer 
with proof masses is enclosed between two capping wafers consisting of sili- 
con blocks embedded in a glass matrix. The electrode metallization on the sur- 
faces of the capping wafers functions as a fixed electrode. Electrodes are fur- 
ther connected to bonding pads via silicon blocks and electrical feed throughs. 
In the fabricated prototypes the total thickness of all three layers is 1.5 mm. 
The accelerometer prototype is shown in figure 5. 

The sensor layout was dedicated for low-g measurement range incorporating 
high accuracy and stability. The prototype sensor has a large active capaci- 
tance value of 3.5 pF and capacitance dynamics of 10% per g for ±2 g version. 
The sensor frequency response is dependent on cavity interior pressure, capac- 
itor gap height and mass surface shape. For the prototype sensor the -3 dB fre- 
quency has been adjusted to 200 Hz. Estimated mechanical noise due to the 
damping is 5 pg/sqrt(Hz). 
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Fig. 4. 3-axis accelerometer prototype. 

The geometrical structure is highly symmetrical and has a rigid frame 
(figure 2), which extends over the whole sensor area. This rugged structure 
protects the capacitors from major deformations by interconnection, tempera- 
ture loads or other external impact due to mounting. Minor temperature 
dependency will exist due to thermal expansion mismatch between silicon and 
glass and due to the dielectric coefficient of the glass. Temperature dependen- 
cy is cancelled to a large degree by the differential capacitance arrangement 
and it is similar in all four differential capacitors due to high level of structur- 
al symmetry. The sensor's shock resistance is improved by several design fea- 
tures. For example protective bumpers limit mass deflection and spring ends 
are rounded preventing high stress concentrations to occur. 





Fig. 6. Model and prototype element hardware on Icent coin 
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The sensor element has many possible interconnection and packaging alter- 
natives. It can be die attached to a substrate or in a housing and wire bonded. 
There is also possibility to directly re-flow solder the accelerometer as an SMD 
component. Flip-chip bonding with proper interconnection bumps would be 
another option that is under discussion. 



4 Manufacturing Process 

A new degree of effective use of silicon area was achieved by implementing 
Deep Reactive Ion Etching (DRIE). This process enables completely vertical 
cuts through the silicon wafer without limitations of silicon crystal orientation. 
Space used for mass separation can be minimised and shapes can be freely 
designed in 2D taking only into account the optimum sensor performance. In 
addition to the DRIE, SOI (Silicon On Insulator) wafers have been introduced 
for this concept. The SOI insulator layer is used here to achieve an efficient 
etch-stop with double-sided DRIE. The thickness of mechanical torsion springs 
is defined accurately by the initial device layer (isolator thickness). The sensor 
element layout was designed to be robust so that even with non-ideal 
processed DRIE trenches the sensor performance will still meet sensor specifi- 
cation. This is due to the fact that the electrical important surfaces (capacitor 
plates) are perpendicular to the trench edge direction. 

The 3-axis accelerometer is partly based on technology choices similar to VTTs 
single axis accelerometers: A 5-layer structure consisting of single crystal sili- 
con and borosilicate glass is prepared separately on one structural silicon 
wafer and two capping wafers. Vertical glass isolations used in capping wafers 
combine thick glass isolation for multiple feed throughs to the hermetic sen- 
sor interior. 

The fixed electrodes are formed by thin film process on both capping wafers. 
The electrodes are connected to the outer contacts by glass feedthrough struc- 
tures. The thick glass layer on both capping wafers will be bonded anodically 
to structural inner wafer with its spring mass system to a hermetic sealed sen- 
sor element. The purely mechanical frequency response of the sensor element 
is adjusted just by the amount of remaining gas fill in the element. Basically 
the final behaviour of the cell is similar to today's VTI sensors in series pro- 
duction. The parasitic capacitances are optimised to extremely low values for 
best sensor performance. 
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5 Signal Conditioning 

The stable properties of the sensor element as the heart of the sensor system 
are the basis for excellent sensor system behaviour. To achieve the required 
performance the sensor interface has to be carefully designed also. The sensor 
signal measurement and processing requires many steps. Figure 6 shows a 
schematic signal flow from real acceleration to digital acceleration signal. Each 
differential capacitance pair is measured by an analog circuit and transformed 
to voltage. Four voltage signals are passed to a block for linear mapping to 
achieve three independent acceleration measurement output signals. 




Fig. 7. Schematic signal flow for 3-axis acceleration measurement. 

For low g-sensors it is sufficient to perform calibration in earth gravity field 
due to the uniform behaviour of the sensor elements. The worst case approxi- 
mation for summed axes mixing and linearity error is 0.1% full scale for ±2 g 
and 0.3% full scale for i6 g. All the linearity and axis mixing calculations are 
based on accelerometer mechanical behaviour and assume ideal signal pro- 
cessing as well as ideal mounting of the sensor element. 

The strength of the concept is that the sensor element and the ASIC are 
processed separately. This leads to the advantage that the signal conditioning 
could be adapted to application demand, like low power consumption, higher 
precision or various analog or digital output formats. 

A failure detection functionality could be added since the system is overspec- 
ified with four masses so that by using that redundancy an unplausible status 
of the sensor could be recognised and flagged-up. Several different strategies 
are under discussion. The main target is to keep the ASIC complexity and size 
low so that cost targets are met. Therefore a straight forward strategy will be 
preferred. 
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6 First Measurement Results with 3-Axis Sensor Element 

Prototypes of the sensors have been manufactured. Each of the eight capaci- 
tors has close to calculated active capacitance level, sensitivity and capaci- 
tance change with acceleration. Figure 8 illustrates the change of each capac- 
itance when the accelerometer is rotated in earth's gravity field around posi- 
tive y-axis in figure 4 starting from the -1 g position for z-axis. Each capaci- 
tance reading is a sum of active and parallel parasite capacitance. 

It needs to be taken into consideration that those output signals were direct 
readings from the sensor element and not subject to signal conditioning. The 
signal conditioning needs to take care of part to part variation in terms of off- 
set and sensitivity adaptations for each single mass. 




Rotation angle (degrees) 



Fig. 8. 8 capacity reading before signal conditioning. 
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7 Packaging 

Different types of applications are underlying different requirements not only 
for the signal conditioning but also for the packaging. The 3-axis element will 
be combined with different type of housings for consumer, industrial and auto- 
motive application. The concept that is mainly addressed to the automotive 
market might look similar as shown in figure 9. The idea is to have a common 
PCB layout or footprint for VTTs new single axis SCA800 sensor as well as for 
the 3-axis version SCA3100 in a slightly bigger housing but with a similar pin- 
out. This would lead to optional use of an appropriate single-axis or multi-axis 
product. A similar solution is already realised with today's SCA610/620 single 
axis family in relation to SCAIOOO family hence this family needs approxi- 
mately 50% more space on the PCB as the next generation of products. 

These housings will fulfil harsh automotive environmental requirements, like 
temperature range from -40 °C to 125°C with an appropriate robustness 
agciinst thermal shock. Both housings are still in draft status so that a final out- 
line will be presented later in the projects. 






SCA800 









SCA3100 



¥¥¥¥¥¥¥ 



Fig. 9. Upgrade compatible laxis SCA800 and 3axis SCA3100 (draft). 
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8 Outlook 

It was demonstrated that the new 3-axis accelerometer concept fulfils the 
requirements set for the sensor element: It has a very high sensitivity and it 
is robust. It shows similar behaviour and performance regarding linearity and 
over temperature stability as it is known from VTI's accelerometer in series 
production as of today. The manufacturing process is feasible and requires 
additionally only one major new process - the Deep Reactive Ion Edging - to 
the existing high volume processes. The sensor element has a small size and it 
is inherently fully hermetic. The rugged design enables to use wire bonding 
connection as well as flip-chipping or even SMT for interconnection. It is seen 
as a major advantage in terms of possible system performance that the novel 
capacitive accelerometer has large capacitance levels and sensitivity com- 
pared with other capacitive accelerometer elements [3, 4]. 

Beneath the development of application specific housings as well as signal con- 
ditioning circuitry a further optimisation of the sensor element towards lower 
height with thinner structures is currently ongoing. It is targeted for applica- 
tions were the height of the sensor element is seen as critical. 
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Highly Integrated Tire Pressure Monitoring Solutions 



M. Osajda, Motorola 



Abstract 

This paper is presenting a new tire pressure monitoring (TPM) sen- 
sor that offers a high level of functional integration by combining a 
0.8 pm CMOS surface micro machined capacitive pressure sensing 
element with a 0.35 pm SmartMOS™ ASIC into a single small outline 
16 pin package. The pressure sensor feature low voltage, low power 
operation, and include an on-chip CMOS radio frequency (RF) trans- 
mitter, a low frequency (LF) receiver, a temperature sensor, a voltage 
sensor as well as embedded motion sensing capability. The data pro- 
cessing is performed by the high logic gate density programmable 
state machine and associated non-volatile memory containing the 
device settings. Several mission profiles and operating modes can be 
programmed to comply either with the US Federal Motor Vehicle 
Safety Standard 138 or with any other existing car manufacturer 
requirements throughout the world. 



1 Summary of US Legal Requirements 

In May 2002, the US National Highway Itaffic Safety Administration (NHTSA) 
officially released the Federal Motor Vehicle Safety Standard 138 (FMVSS138), 
Tire Pressure Monitoring Systems, Controls and Displays that requires the 
installation of tire pressure monitoring systems to warn the driver when a tire 
is significantly under inflated. 

Application - Passenger cars, trucks, multipurpose passenger vehicles, and 
buses with a gross vehicle weight rating of 10000 pounds (~4536 kg) or less, 
except those vehicles with dual wheels on an axle. 

Compliance Option - Both Direct and Indirect system will be allowed during 
the phase-in period from November 2003 through October 2006 and with the 
following technical requirements. (Note: A US federal court decision dated 
August 6, 2003 will prevent the car manufacturer to implement indirect sys- 
tem as currently specified by NHTSA. 
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The agency has decided to revise the standard. At the date of writing this 
paper, the new standard was not released). 

Indirect Measurement - Indirect TPM must warn the driver when the pres- 
sure in any single tire has fallen to 30% or more below the vehicle manufac- 
turer's recommended cold inflation pressure for the tires, or a minimum level 
of pressure specified in the standard, whichever pressure is higher. 

Direct Measurement - Direct TPM must warn the driver when the pressure 
in any single tire or in each tire in any combination of tires, up to a total of four 
tires, has fallen to 25% or more below the vehicle manufacturer's recom- 
mended cold inflation pressure for the tires, or a minimum level of pressure 
specified in the standard, whichever pressure is higher. 

Phase in Schedule - Specified in percentages of vehicles sold in the USA 
(table 1). 




November 2003 th-ough October 2004 min 1 0% 



November 2004 through October 2005 min. 35% 

November 2005 through October 2006 min 65% 

After Ncnember 2006 100% 

Tab. 1. FMVSS138 USA Phase In schedule. 

There are currently no legal requirements outside the USA, but thanks to the 
added safety provided by such a system, most of the car manufacturer around 
the world have decided to implement TPM on most of there new platforms. 

The next paragraphs of this paper will present a solution dedicated to direct 
pressure monitoring. 
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2 A New Highly Integrated Tire Pressure Monitoring Sensor 

2.1 Technical Requirements and Challenges 

The implementation of the FMVSS138 in the USA has driven the automotive 
and semiconductor industry to a new level of integration and cost reduction 
rarely achieved in this industry. In order to respond to this new requirement. 
Motorola has developed a new device with all the required TPM functionalities 
embedded in a single surface mount package. 



2.2 TPM Required Set of Features 

The following feature set is what is commonly required by the TPM system 
manufacturer: Absolute pressure measurement, temperature measurement, 
voltage measurement, motion detection, embedded signal conditioning and 
signal processing, access to multiple datagram format, possibility to commu- 
nicate with the device via an LF channel, embedded RF transmitter, ultra low 
power consumption and cost effective. 

The challenge for Motorola was to create such a device by combining the 
appropriate technologies in a single package, and being able to produce it in 
very large volumes. The following approach has been taken. 



2.3 The SmartMOS™ ASIC, the Brain of the System 

The basic functions to be performed in a TPM module are relatively straight 
forward: measuring pressure, measuring temperature, and transmitting both 
data and a serial number. But to support this basic function there are several 
subsystems or modules that have their own distinct events to analyze. TPM 
system enhancements may also bring other modules into consideration such 
as measuring the battery voltage or ability to communicate bi-directionally 
with the TPM system. And finally, some modules may be introduced with the 
sole intent to reduce power consumption. Figure 1 shows a typical architecture 
of a wheel module with the required functions. 

Pressure Measure (1) - The event must power up the sensing element, con- 
vert its variation to a voltage, store that result using a sample and hold capac- 
itor and then shut down. 




26 I Safety 



Temperature Measure (2) - The event must power up the sensing element, 
convert its variation to a voltage, store that result using a sample and hold 
capacitor and then shut down. 

Voltage Measure (3) - The voltage measurement is primarily to monitor the 
hattery. It is done using a voltage threshold comparator for the minimum bat- 
tery voltage. There must also be a ftiirly accurate voltage reference. This event 
must power up the voltage reference and the comparator, store the result and 
then shut down. There will also be a requirement to perform this voltage 
measurement during both the highest current event (usually the RF transmis- 
sion) and during any event requiring the highest battery voltage (usually an 
analog measurement). 

A/D Conversion (4) - If any of the prior measurements create an analog volt- 
age output, then an A/D converter will be required to convert the voltage to a 
digital code for further processing. This event will be to power up the A/D con- 
verter, convert a stored voltage, store the resulting data, and then shut down. 




Fig. 1. Wheel module architecture. 

Data Processing (5) - The data collected from the above measurements must 
be analyzed according to a defined operational algorithm. The result will 
determine how the TPM will operate in the future and whether a transmission 
is to be made. The data processing will also format the data for the RF trans- 
mission. The power consumed by such digital logic will depend on a very small 
standby current term and a speed-related term that is linearly related to its 
clocking speed. Further, each data process can be separated out into individual 
events such as read data, calculate data, format data, send data, etc. 
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Wake Up (6) - The wheel sensor will normally be in a standby mode waiting 
for a need to measure and process data. This wake-up function is usually a very 
low power oscillator that outputs a pulse every 3 to 6 seconds. This event will 
be one of the long time standby current events that may exist for the complete 
life of the battery. 

RF Transmitter (7) - The RF transmitter module is a combination of some dig- 
ital processes and the actual RF power transmitter. This is the highest current 
consumption module and creates the highest current pulse. However, while 
transmitting the operational voltage may be less than required for the meas- 
urement modules. Therefore the battery cut-off voltage will be different from 
that for the other modules. In all cases, a minimal number of RF transmissions 
are the most important feature of the operational algorithm. A confounding 
problem with the RF transmitter is how to handle the possibility of simulta- 
neous transmissions from multiple wheel sensors. This would result in data 
"collisions" causing the RF receiver to be unable to discern any data. The usual 
method to solve this issue is to randomize the transmission times and also send 
multiple randomized data words within any given transmission. 

NVM Memory (8) - The non-volatile memory (NVM) module serves several 
purposes. One purpose of the NVM is to store trimming values to make each 
sensor meet the required accuracy for pressure and temperature measure- 
ments. One other purpose for the NVM is to store a unique serial code for each 
wheel sensor and to store the configuration bit of the device. The NVM may 
draw current for each programmed bit as well as some digital switching cur- 
rent during its access. 

LF Receiver (9) - One method to identify and/or wake up the wheel sensor is 
to transmit a low frequency electro-magnetic signal to it. This is usually done 
at frequencies around 125 kHz using a transmitting coil mounted on the chas- 
sis or axle housing somewhere near each wheel and a small LF receiver coil 
within each wheel sensor. The LF receiver can also be used to trigger a given 
wheel sensor to transmit within a given time frame. In this way the chassis 
receiver can match up the wheel sensor's serial ID with the location that was 
triggered and provide tire localization information. A further use of the LF 
receiver would be to receive test and other commands from a closely placed 
inductive transmitting coil. This can provide diagnostics and fast identification 
of wheel sensors in either the factory or field. 

Over-Temperature (10) - A concern with TPM is that extremely high tem- 
peratures may develop within the tire and wheel, which are beyond the nor- 
mal operational range for the wheel sensor components. The major source of 
such high temperatures is the braking system that can transfer heat through 
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the hub to the wheel rim. The concern is whether the wheel sensor may start 
to operate erratically above 125°C and either transmits erroneous data about 
the tire or starts transmitting continuously and therefore drains the battery 
unnecessarily. Some sort of over-temperature switch is needed to prevent data 
transmission above 125°C. 

Power Management (11)- The most critical operational issue with TPM is 
keeping total power consumption to a minimum. Therefore, power manage- 
ment requires special attention. This could be accomplished by simply provid- 
ing on/off enables in each module and controlling these enables with a small 
microcontroller unit (MCU). The drawback of this approach is that the MCU 
requires a number of instructions to manipulate the enables. A simple change 
of one control line or a logical AND or OR function may consume multiple cen- 
tral processing unit (CPU) bus cycles. If such power control sequences were 
controlled by logic the changes would be nearly instantaneous and consume 
much less power. Switching of a number of logic nodes (such as in an MCU bus) 
takes more power than switching a single NAND gate in a state machine. 

Therefore as many tasks as possible should have their power sequenced using 
hardware logic in small state machines. The implementation of those small 
state machine is possible by using a process such as SmartMOS™, a process 
enabling the combination of analog, digital, NVM and RF functionality on a 
same die as shown in figure 2. 




Charge Pump 



8-Pit CPU 

HC08, 



Analog MOS 

to 40V 



RAM, MCU 

256BRAMI, SPI. SCIetc 



A D dO bit) 

SV, comefsen 



Osoillatof 

Crytttl. • toSUHs, 2 % 



FPROM 

10KB ROM 



d.Bit D-A 

CuMenteed monotone 



Power MOSFETs 

to40V. 2A 25amO 



High Speed CMOS 

a.av, lOK'sofgotos. • 3SMHz 



Independent 
Thermal Management 

Shutdowi 9(1185. ±15 C 



Comparators 

5rnV, or autoz9ro«d 



Load Sensing 
Stioned LMd 
Open Load 
PirMte Movement 
Lc«d Reelstaroe 
induct (Vice 



Precision References 

Bandgep - 1 .2V etfCetc. 



Voltage Regulators 

MCU Supplies. SV, 3.3V. 2.6V etc. 



CMOS 

UHF PLL Transmitter 



Fig. 2. Capabilities of SmartMOS™. 
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2.4 The Surface Micromachined Sensing Element 

The primary goals of the transducer design are the following: 

1) Produce a sufficient base capacitance 

2) Minimize pressure linearity errors 

3) Maximize pressure sensitivity 

4) Minimize the die area 

5) Minimize temperature errors 

6) Minimize the parasitic capacitance to the substrate. 

'Ifaditionally, there has always been a trade-off between items 2 and 3; greater 
sensitivity usually produces larger linearity errors. Although linearity errors 
can be improved with reduced pressure sensitivity, more circuit gain is 
required, and circuit noise and other errors can increase. Thus, there is a 
desire to have a high-pressure sensitivity and a low-pressure linearity error, 
and therefore, other approaches were investigated to reduce linearity errors. 




Fig. 3. Transducer using an internal post diaphragm design(a). 
Transducer using a rectangular diaphragm design (b). 
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Finite element simulations were used to evaluate candidate diaphragm shapes 
for uniformity of diaphragm deflection. In addition to previously used shapes, 
such as circles and squares, "internal post" configurations were evaluated. 
Internal post designs anchor the diaphragm both at its perimeter and at inter- 
nal locations. Simulations showed that some configurations of internal posts 
could produce more uniform diaphragm deflection, and significantly reduce 
linearity error at a given sensitivity level. Figures 3(a) and 3(b) show photo- 
graphs of an internal post design and a rectangular diaphragm design. 

The methodology used in the insertion of the sensor module into the CMOS 
process was based on the requirements that there is no impact, either ther- 
mally or topographically, on the CMOS fabrication. This is done by insertion of 
the micromachining steps before the temperature sensitive process steps of 
the CMOS process. It was identified that the source/drain implants and the 
subsequent thermal processing were the temperature sensitive steps. In addi- 
tion, the precess integration was designed to reuse, as much as possible, the 
fabricatien steps of the CMOS process. The cross-sectien of the CMOS and sen- 
sor devices is shown in figure 4 illustrating the key features of the process inte- 
gration. 




(b) Sensor isolation nitride and spacer 




(d) Resume CMOS standard process with 
Source/Drain formation 



Fig. 4. Selected cross-sections of CMOS integrated capacitive pressure 
sensor process flow. 
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The fabrication process starts with the standard CMOS fabrication process in 
which the active areas are defined using an isolation field oxide using a LOCOS 
process. In addition to defining the active area of the CMOS devices, the field 
oxide is used as the area where the sensor is to be formed. The next step is the 
formation of the floating gate of the EEPROM (note: EEPROM may be imple- 
mented either on the sensing element or on the ASIC, depending on the sys- 
tem partitioning). The required extension implant of the EEPROM is complet- 
ed followed by the deposition of the thin tunnel oxide. The EEPROM floating 
gate is then deposited using a doped LPCVD polysilicon film. The floating gate 
polysilicon is patterned and etched so that it also forms the bottom fixed plate 
of the capacitive pressure sensor. The isolation of the fixed electrode to the 
substrate is provided by the CMOS field oxide isolation. The CMOS transistor 
gate oxide is then deposited followed by the gate polysilicon deposition. 




Fig. 5. CMOS Integrated pressure sensor. 

At this stage, the CMOS fabrication is suspended and the sensor fabrication 
steps are inserted. The first step is the deposition of a layer of PECVD oxide to 
protect the CMOS gate Poly from future sensor processing. The next step is the 
removal of the PECVD oxide, gate polysilicon and the gate oxide from the sen- 
sor area. This is done in such a way that the bottom polysilicon plate is not 
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affected by the etching process. After the gate oxide is removed, an isolation 
layer of low stress silicon rich nitride is deposited using an LPCVD process. The 
silicon rich nitride is chosen for its excellent etch resistance to HF based 
chemistries. The silicon nitride is used to isolate the top and bottom plates of 
the pressure sensor and also protect the CMOS area from subsequent micro- 
machining steps. 

The sacrificial layer of the sensor is next deposited using a phosphorus-doped 
glass. The thickness of this layer defines the spacing between the bottom fixed 
electrode and the moveable diaphragm. The spacer layer is also patterned for 
transducer designs, which utilize internal support posts for improved pressure 
linearity. The sensor diaphragm is then formed by deposition of a LPCVD poly- 
silicon layer. The polysilicon is then doped, patterned and etched to define the 
diaphragm. 

The pressure sensitive diaphragm is then released by removing the sacrificial 
layer below the diaphragm polysilicon. Etch ports located at the periphery 
allow the HF based chemistry to completely remove the phosphorus-doped 
glass. The etch ports are then sealed using a line of sight sealing with a PECVD 
deposited oxide layer. The thickness of the PECVD oxide is chosen to ensure 
that the etch holes are plugged and the cavity between the fixed plate and the 
diaphragm is sealed at the deposition pressure of the PECVD oxide. This com- 
pletes the formation of the sensor module. 

The next step is the resumption of the CMOS fabrication process. This is done 
by first removing the sealing oxide, isolation nitride and protective oxide over 
the CMOS device area. The CMOS device formation is then resumed with the 
formation of the source/drain regions, and the subsequent thermal treatment 
to provide the activation anneal and definition of the junctions. This thermal 
treatment also serves to activate the dopants and provide the stress relief 
anneal in the sensor polysilicon film. 

The first interlayer dielectric layer is next deposited, patterned and etched to 
allow for the subsequent first metal layer to contact the various CMOS devices 
as well as provide the interconnect for the sensor to the circuit. The second 
interlayer dielectric layer is then deposited, patterned and etched to form vias 
for the second metal to contact the first metal with the routing required for 
the circuit blocks. Finally, the passivation layer is deposited over both the 
CMOS and sensor device area. The passivation layer is then patterned and 
etched to expose the bond pads. In addition, the passivation layer, second, and 
first dielectric layers are then removed over the pressure sensitive element but 
not from the reference sensor element. This exposes the polysilicon 
diaphragm of the sensitive element leaving it free to deflect under an applied 
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pressure. The reference element has very little deflection since it has a very 
thick composite diaphragm. Both the pressure sensitive element and the ref- 
erence elements are used hy the circuit to generate the output of the sensor. 
This completes the description of the process integration of the capacitive 
pressure sensor with the CMOS devices. Figure 6 shows a die picture of a 
CMOS integrated pressure sensor. 



3 The Combination of a Sensing Eiement and Dedicated ASIC in a 
Singie Package 

3.1 Block Diagram 

Figure 6 shows the block diagram of the new sensor with all the previously 
described features integrated on two different chips: A surface microma- 
chined sensing element and a SmartMOS™ ASIC, both packaged into a dedi- 
cated SOIC16 package. A third chip, more specifically a piezoelectric ceramic 
can be included in the package to provide an embedded motion detection (not 
shown and not further described in this paper). 




HOV LF SLD CLK DRV 



Fig. 6. Block diagram of the new sensor. 





34 I Safety 



3.2 Device Characteristics 



1 Operation Range I 


Supply Voltage 


2.1 to 3.4 V 


Continuous Temperature Range 


-40 to 125°C 


Temperature Excursion < 10 min 


-40 to 150°C 


Sensors Characteristics 


Pressure Range 


50 to 450 kPa or 50 to 900 kPa ab& 


Pressure Accurancy (*) 


-i-/-7kPato +/- 15kPa 


Temperature Range 


-40 to +125°C 


Temperature Accurancy 


-i-/-4“C 


P, T and V resolution 


8 bits 


Communication Interface I 


RF Data Encoding 


Maiidiestci or Bi-Phasc 


RF Modulation Schema 


ASKoiFSK 


RF Baud Rate 


4.8, 9.6 & 19..2kbs 


RF Output Power 


-1 dBm or +5 dBm 


LF Input Sensitivity 


Up to 2 mVpp 


LF Input frequency 


125 kHz typ. 


Power Consumption 


Standby (*) 


1.1 uAtyp 


During Measurement (P,V or T) 


20 uA sec typ 


During Transtrission (9.6k) (*) 


200 iiA sec typ 


Peak Current (*) 


8 m A typ 


Package 


Centrifugal Forcce Resistance 


2000 g 


Unpowered Shock Resistance 


2000 g 


Maximum Pressure (absolute) 


1200 kPa 


Package 


Media Resistant, Integrated Ait Filter 



Tab. 2. Device characteristics. 



3.3 Flexibility Provided by the State Machine 

A significant number of NVM bits in the on-board memory are allocated for 
the customization of the device, allowing the customer to select exactly the 
required operating mode. The following customization options have been cur- 
rently implemented in the new sensor: 

► 4 mission profiles, each defining a set of timing between the different 
states. 

► 15 transmit mode in order to address the different receiver configura- 
tions. 

► 2 modulation frequency bands to address the different RF regulation 
world wide. 
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► 2 RF output power options to optimize the power consumption per car 
platform. 

► 2 datagram format options. 

► 2 data frame bit length options (82 bit / 90 bit). 

► 2 data encoding schemes, Manchester or Bi-phase. 

► 2 data polarity options (normal / reversed). 

► 2 LF sensitivities options, high and low. 

► 2 averaging methods, to minimize noise. 

► 4 delta pressure algorithms, defining the conditions to enter into alert 
mode. 

► 4 preamble options to wake up properly the receiver. 

► 2 motion detection options (enable / disable). 

► 2 over temperature options (enable / disable). 



In total, more than 300 000 combinations are possible within a given state 
machine to address the most demanding customer specification. 



C1 

tbd 




Fig. 7. Application schematic example. 



3.4 Application Example 

Thanks to the high level of integration and extremely low power consumption, 
it is quick and easy and for the end user to build a cost effective wheel mod- 
ule (figure 7). The new sensor is delivered in a small outline 16 pin package 
and only a few external passive components, low frequency coil and crystal are 
required. Since there is no software development required, time to first wheel 
module prototype is significantly reduced. 
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The reduced number of external components also greatly enhanced the relia- 
bility of the wheel module by reducing the number of electrical connections 
and solder joints. The overall size and weight of the wheel module can be 
reduced to dimensions never before achieved. 



Conclusion 

Motorola's new dedicated Tire Pressure Monitoring Sensor is a unique oppor- 
tunity for the automotive industry to implement a cost effective and reliable 
semiconductor solution inside the tire for continuous and accurate tire moni- 
toring, thereby enhancing the overall safety of the car and its occupants. The 
combination of surface micromachined MEMS technology and CMOS ASIC is 
paving the way for future battery-less TPM solutions. Several battery-less 
architectures are under investigation by the industry (125 kHz transponder 
technology, UHF energize and back scatter modulation, solid-state batteries or 
even micromachined generators embedded in the tire). However, none of them 
can be efficiently implemented if the heart of the system consumes too much 
energy! 
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Abstract 

Advanced automotive sensor systems used for tire pressure meas- 
urements and motor management show a strong demand for 
increased robustness, functionality and flexibility at reasonable low 
costs. To meat these requirements a single-chip sensor in a low cost 
package has significant advantages compared to other existing solu- 
tions. By combining silicon micro-machining designs and processes 
with advanced mixed-signal CMOS circuitry, many of the early limi- 
tations of single-chip sensors can be overcome. The result is a high- 
performance single chip pressure sensor that provides an amplified 
and calibrated output. Co-integration of sensor, CMOS amplification 
and EEPROM on a single chip results in a sensor structure that can 
be completely moulded into the package prior to calibration and 
compensation. The calibration and compensation parameters are 
then programmed into the package on an automated tester, provid- 
ing a very low-cost high-volume sensor element. 



1 Background 

Piezoresistive pressure sensors need signal conditioning circuitry in order to 
be interchangeable and compatible with most electronic control systems. Each 
sensor element has individual characteristics and is sensitive to both pressure 
and temperature. The signal conditioning functions include calibration for off- 
set and full scale variations to make each unit electrically interchangeable, 
temperature compensation for offset and full scale, and sometimes other func- 
tions as well, such as linearity correction, diagnostics, and filtering. 




Ti’aditional technologies to marry signal conditioning electronics with sensing 
elements have resulted in technical and commercial compromises in product 
cost and performance. 



Parameter 


Two-Chip 

Thick-Film 

Laser 


Two-Chip 

Thin-FUm 

Laser 


One-Chip 

Thin-ftim 

Laser 


One-('hip 
. inalogue 
Correction with 
LLPROM 


One-('hip 

Digital 

Correction with 
LLPROM 


Circuit Complexity 


Low and Low 


Low and 
Moderate 


Moderate to 
High 


High 


High 


Process Complexity 


Low 


Moderate 


Moderate to 
High 


Moderate to High 


Moderate to 
High 


Component Costs 




High 


Low 


Low 


Low 


Test Complexity 


High 


High 


High 


Low 


Low 


Test Cost 


Very High 


High 


High 


Low' 


Low 


Package Size 


Large 


Ciood 


Small 


Small 


Small 


Package Cost 


Very High 


High 


Low 


Low 


Low 


Pressure Range 
Adaptability 


Good 


Good 


Moderate 


Moderate 


Moderate 


Trim after get and/or 
packaging 


No 


No 


No 


Yes 


Yes 


Ability to do multi- 
order correction 


No 


No 


No 


Yes, with 
difficulty 


Yes 


Accuracy 


Low 


High 


Moderate 


Moderate 


Very High 


Resolution 


High 


High 


High 


High 


Moderately 



High 



Tab. 1. Comparison of various signal conditioning and error correction 
approaches. 

Monolithic approaches have resulted in low cost and small size but in many 
cases have compromised device performance due to the restrictions place on 
circuit and sensor design by the monolithic fabrication process. Hybrid 
approaches use either a dedicated ASIC for signal conditioning or a discrete 
circuit approach. In many cases, these approaches have better performance 
than monolithic devices and offer flexibility for adapting new designs with 
simple components changes. However, these hybrid approaches are generally 
not as low in cost as the monolithic approaches and are also larger and. require 
additional assembly steps. More electrical and mechanical connections means 
more concerns about reliability. A comparison of different approaches is 
shown in table 1. 
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2 Applications 

There are a number of applications such as Manifold Absolute Pressure (MAP) 
sensing and Tire Pressure Monitoring (TPM) that are driving the demand for 
integration as a way to lower system costs. In these automotive applications, 
especially, the need is for an integrated system design of sensor, signal pro- 
cessing and packaging together in order to realize the optimal solution. 

There are a number of technologies to store the calibration and compensation 
values for each sensor ranging from thick film resistors that are laser trimmed 
after device testing to achieve temperature compensation and calibration to 
electronic trimming where the compensation and calibration values are elec- 
tronically programmed into the chip - either with a one-time programmable 
technology such as fuses, zener diodes, on-chip trimmable resistors, or a re- 
programmable technology such as EEPROM. 

Re-programmable technologies such as EEPROM allow coefficients to be pro- 
grammed into the device multiple times, allowing for real-time programming 
during manufacturing or programming after assembly based on the test data 
for a given device. 



3 Co- Integrated Approach 

The co-integrated pressure sensor was collaboration between MEMS sensor 
designers and mixed-signal IC designers. This collaboration brings the 
strengths of each technology to the device design to maximize performance. 
The circuitry performs calibration for offset and sensitivity, and multi-order 
temperature compensation and error correction. On-chip EEPROM stores the 
calibration coefficients for the device. The manufacturing process flow has 
been designed to allow all standard CMOS steps to occur at the front-end of the 
process, with the micro-machining steps performed after the circuitry is com- 
pleted. 

The result shown in figure 1 is based on a standard 0.65 gm mixed-signal 
CMOS process with EEPROM. All of the CMOS processing is performed at the 
start of the process and the MEMS processing is performed at the end. This 
process flow results in fewer compromises on the circuitry and better sensor 
performance. Current chip size is 10 mm^. 

In addition, this "CMOS first" choice results in a better manufacturing flow 
because the CMOS process flow remains uninterrupted. With this approach. 
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process controls, based on high volume production, are maintained. Sensor 
design has incorporated readily available standard P-type implanted sensing 
elements, eliminating the need to vary the process to incorporate an extra, 
special implant. 

The MEMS process that follows the CMOS process steps does not adversely 
affect the CMOS functions. These processes include the silicon etch to form the 
pressure sensitive diaphragm, anodic bonding of a glass substrate for absolute 
pressure configurations, probing, wafer dicing, and inspection. 



Digital Signal Processor 
(DSP) 



Oscillator 



Sensor 



EE 

PROM 



D-to-A' 
Converter 
and Ouqgiut 
Stage 



Input Stage 
and A-to-D 
Converter 



Fig. 1. Photomicrograph of co-integrated pressure sensor. 



4 Technical Details 

The circuit on the co-integrated pressure sensor provides for a number of 
adjustable parameters. Based on production test data, the following parame- 
ters can be adjusted in the ASIC by programming the device using the on- 
board EEPROM. 

► Pre-amplifier Gain - 32 to 152 in 16 steps (16 to 76 at 2.5 V offset). 

► Pre-amplifier Gain sign - 1 bit (positive vs. negative output). 

► Pre-amplifier Offset - 0 to 49.5 mV in 16 steps. 

► Pre-amplifier Offset sign - 1 bit (positive vs. negative input). 

► Upwards of 20 coefficients in Pressure and Temperature correction. 

► Output Clamp - Independent Adjustment in 40 mV steps. 

► Output Filter - 1 to 256 Conversions in 1, 2, 4, 8, 16, 32, 64, 128 and 
256 steps equivalent to approximately 1 to 256 mS. 
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4.1 Circuit Block Diagram 

The circuitry on the co-integrated pressure sensor includes all of functions for 
signal conditioning and calibration including amplification, correction, span 
calibration, temperature compensation, and multi-order non-linearity correc- 
tion for pressure and for the temperature coefficients. A block diagram of the 
circuit is shown in figure 2. The system uses an 11 -bit A/D converter with 8x 
over-sampling to provide for an effective 14-bit data conversion resolution. 
Data is then processed thru an on-board Digital Signal Processor (DSP) where 
the A/D data is corrected, based on stored calibration coefficients stored in the 
on-board EEPROM. The corrected signal is then fed to a 12-bit D/A converter, 
which in turn drives the output amplifier. The amplifier has been designed to 
be able to drive > 2 nF of capacitance as needed for EMF suppression in the 
automotive environment. 




Fig. 2. Block diagram of circuitry on co-integrated pressure sensor. 
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4.2 Calibration 

Calibration is done by measuring the sensor at multiple temperatures and 
pressures. Uncorrected data from the A/D converter is read out thru a digital 
I/O for each data point. The external calibration computer then determines the 
minimum error and loads the order of the correction in pressure and temper- 
ature and correction coefficients into EEPROM. Verification of the calihration 
can then be performed to verify calibration accuracy as desired. 



Best Fit Straight Line (BFSL) Linearity 
M777.01B with 015-A Sensor 
Sensor 3 [p4 x t3] 




Temperature, "C 

♦ Uncorrected Digital Pressure — •— Corrected Digital Pressure Corrected Analog Pressure 

Fig. 3. Linearity correction capability of co-integrated sensor system. 

Figures 3, 4, and 5 show data from sensors before and after correction by the 
circuitry. The data in figure 4 shows the ability of the signal processor to com- 
pensate for pressure non-linearity, over at temperature range of -20 °C to 
-i-110°C. 

As shown in figure 4, the signal processor can also compensate to hotter than 
0.2% for temperature coefficient of zero error, even when the sensor exhibits 
a 2'''^ order non-linearity in the temperature coefficient of zero. 
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Figure 5 represents the corrected temperature coefficient of span. The -20% 
change with increasing temperature of the uncompensated part gets correct- 
ed to close to zero. Total error over the compensated range for this sensor-sys- 
tem is less than 0.25%, including all errors combined including initial zero and 
full-scale gain-set errors. 



Tefii|>er«tvr« Coefficient of Zero (TCZI wrt 2$'C 
M777 01B witli OtS^ Sen»or 
&eo»or3 |»< xO| 



06 




.06 



.40 .20 0 20 40 60 60 100 120 

Temperature, *C 

6 UnMireited OiqW Pieiauie -*^~Corl«eM Optel nre*si»e 4 Cefreded Aneloy PieeMrt 

Fig. 4. Temperature coefficient of zero over temperature. 

Over temperature, the performance of the device can change and thus cali- 
bration at other temperatures means increased accuracy at those tempera- 
tures. Examples of the ability of the DSP to correct errors in the sensor's out- 
put are shown in figures 3, 4 and 5. As can be seen in the figures, the algo- 
rithm used can be adapted to easily correct pressure non-linearities of various 
orders and further can facilitate correction that might be introduced by tem- 
perature dependent pressure non-linearities from gelling or other media inter- 
faces. 
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Fig. 5. Temperature coefficient of span over temperature. 



4.3 Sensor Construction 

After completion of the CMOS processing steps, the wafers are shipped from 
the CMOS fab to the MEMS fab. The silicon etching is performed along with 
the final process steps for wafer bonding, dicing, and testing. The resulting 
die, a cutaway-cross-section of which is shown in figure 6, are then either 
shipped to die customers or assembled into a variety of packages. 
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Fig. 6. SEM cross-section showing CMOS and sensor area and bottom 
glass wafer. 



4.4 Improvement over Two-Chip Solutions 

By incorporating the DSP-based correction engine capable of multi-order cor- 
rection for pressure and temperature errors, much better accuracy can be 
achieved with the co-integrated sensor than with a two-chip solution. For 
example, many two-chip solutions have good accuracy over temperature 
ranges near room temperature, but then have increasing error as temperature 
reaches the temperature extremes. With the co-integrated sensor, the error 
curves can remain the same over the entire temperature range. 



4.5 Packaging 

Co-integrated piezoresistive pressure sensors with digital correction engines 
and EEPROM calibration data storage have been reported before [1]. But up to 
now cost efficient packaging in the most used 1C packaging technology plastic 
overmoulding failed due to different problems: packaging induced stress, ther- 
mal drifts and overstress caused by the moulding pressure [2]. By using a 
patented injection moulding approach [3] the packaging for the presented 
pressure sensor system is done in a 16-pin SOIC cavity overmoulded package. 
The special technique allows to expose the diaphragm area of the sensors chip 
as shown in figure 7, the wire bonds are overmoulded as in a standard IC pack- 
age with the same level of proven reliability. Any stresses from die attach- 
ment, wire bonding, and moulding are compensated by programming the trim 
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parameters in the final package. The result is a high performance part. This 
basic package is a foundation for other configurations, including further gel 
filling of the cavity. Various ports may also he welded on the package. 




Fig. 7. Single chip pressure sensor in SOiC 16 plastic package. 

After programming, the EEPROM is electronically locked as a completed mod- 
ule, or further trimmed after system assembly. Because of the use of EEPROM, 
the trim parameters can be changed after initial programming, meaning that 
the trim values can be optimized during test or trimmed again after assembly 
of the sensor package into the system (for example, for serialization informa- 
tion or fine offset correction). 

The resultant system essentially combines three technologies together to pro- 
vide the user with a cost-effective solution: 

► ASIC signal processing - ELMOS Semiconductor AG. 

► Silicon micro machining and Test - Silicon Microstructures, Inc. 

► Specialized Plastic Lead-frame packaging - Eurasem b.v. 



5 Conclusion 

A new, single-chip pressure sensor has been built with all signal conditioning 
and pressure sensing functions on the same chip. This co-integrated pressure 
sensor overcomes many of the performance limitations of previous single chip 
designs. 
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The co-integrated pressure sensor offers improved performance over previous 
single chip pressure sensor solutions while reducing size and cost compared to 
hybrid configurations. A process flow that is compatible with a standard 0.65 
micron CMOS process has been used along with MEMS processing to create 
the pressure sensor with EEPROM on-chip to store the calibration and com- 
pensation values for the sensor. Test results show a dramatic improvement in 
device performance over the uncompensated result. 

This co-integration technology can be extended beyond pressure sensors to 
other devices as well, including accelerometers, gyros, and other sensor types. 
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Abstract 

Rollover accidents often result in heavily injured or even killed pas- 
senger. Most often head injuries occur as a consequence of heads 
contact with the roof of the vehicle or the road when the head pen- 
etrates through the side window during rollover. Therefore industry 
is focusing more and more in rollover detection and rollover systems. 
In this paper new concepts and new algorithm design for vehicle 
rollover detection systems are presented. Delphi developed a fully 
tested system, which allows detecting rollover scenarios to trigger 
restraint systems as rollbars, belt pretensions, window airbags 
and/or head airbags. 



1 Rollover Risks and Statistical Background 

Rollover accidents are very heavy accidents. But different statistics are exist- 
ing about rollover accident analysis. One reason for statistical variation is that 
these accidents are not counted as separate accidents. The police does not have 
separate sheets for statistical rollover analysis. As a consequence the number 
of cars in the statistic of a country is sometimes very low. The other reason is 
that a rollover usually does not occur alone. Most often a crash is followed by 
a rollover or vice versa. If one focuses on statistics with high number of acci- 
dents in different countries you can find also differences due to different vehi- 
cle population in a country and different street styles. For example in the 
United States you can find more SUVs (Sport Utility Vehicle) than in Europe or 
Asia. SUVs more often than other vehicles tend to roll because of the high cen- 
tre of mass. Also in Eastern part of Germany there are more avenues than else- 
where. In case of a crash a car often hits a tree and sometimes tends to roll 
after this collision. In Great Britain a lot of detailed analyses about rollover acci- 
dents have been performed. The statistical data from British Police is very 
extensive and detailed. So this database can be used as a standard for Europe. 
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1 .1 British Accident Data 

British national data are commonly called "STATS 19" [1] due to the name of 
the form that the Police officers fill in for every road traffic accident involving 
an injury on a public highway. In this statistic about 200.000 accidents are col- 
lected per year. There are three different types of records: accident, vehicle 
and casualty records. An accident record is completed for each accident. A 
vehicle record is completed for every vehicle involved in the accident (even if 
no person was injured). The casualty record is completed for every injured per- 
son in the accident. The overall criteria for an accident to be included in the 
STATS 19 records are that a person must have been injured in an accident on 
a public highway. The data are available approximately 14 month after the 
year of collection in a form ready for electronic analysis. 




Fig. 1. Rollover accident in USA. 

In figure 2 it can be seen that 19.3% of all Killed or Seriously Injured (KSl) per- 
sons are involved in rollover. This is even more than the rate without any ele- 
ment of roll (e.g. two cars crash front - front). So the highest rate of KSI acci- 
dents are rollover accidents. In 1999 in GB 2320 people died or have been seri- 
ously injured in cars with element of roll. In the British database a lot of very 
detailed information can be found (e.g. population characteristics, circum- 
stances of accident, loading and road type as well as environmental conditions 
at the time of the accident. 
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1 .2 US Accident Data 

From 1999 to 2001 in USA the number of passenger vehicle occupants killed 
in all motor vehicles crashes increased by 4%, while fatalities in rollover crash- 
es increased by 10%. In the same decade, passenger car occupant fatalities in 
rollover declined by 1 5 % while rollover fatalities in light trucks increased by 
43%. 



Percentage of Casualties With 
Fatal or Serious Injuries (KSi-Rate) 




Cars - no element of Cars - element of roll All Cars 

roll 



Fig. 2. Percentage of casualties with fatal or serious injuries [1]. 

In the United States of America the U.S. Department of Ttansportation pub- 
lishes an annual report [2]. In this document about 6.3 million accidents are 
reported per year. About 2 million of them are KSI types. Figure 3 shows one 
of the graphs out of the annual report of 2001. It can be seen that the risk of 
a rollover for pickups and light trucks SUVs is much greater than for vehicle 
cars. 19% of all fatal crashes are with element of roll. More detciils can be found 
in the report "Characteristics of Fatal Rollover Crashes" [3]. In U.S. rollovers 
account for 20% of passenger vehicles in fatal crashes. This is almost the same 
than in Great Britain. Due to this high amount of KSI accidents occur in com- 
bination with rollover (especially with SUVs) there are discussions about 
rollover safety systems for new vehicles. Especially for SUVs there is a lot of 
discussion going on. 

To be able to compare useful rollover safety strategies it is important to define 
rollover resistance measuring methods. Therefore the NHFSA has done a so- 
called "New Car Assessment Program". The goal of this program was to find 
rules for the customer to assess the resistance of a car against rollover. In [4] 
this report can be found. NHTSA proposed to combine a vehicles Static 
Stability Factor (SSF) measurement with its performance in the so-called 
"Fishhook" manoeuvre. The Static Stability Factor of a vehicle is defined as the 
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ratio of one half its track width to its center of gravity height (see 
www.nhtsa.dot.gov/hot/rollover/ for more ratings and explanatory informa- 
tion). The predicted rollover rate will he translated into a five-star rating sys- 
tem that is the same as the one now in use: One star is given for a rollover rate 
greater than 40%; two stars for rates between 30 and 39%; three stars for 
rates between 20 and 29%; four stars for rates between 10 and 19% and five 
stars are given for 10% or less. 




Fig. 3. Percent reliever occurrence by vehicle type and crash severity [2]. 
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U.S. light truck rollovers by crash severity. 

Source: NCSA, NHTSA, FARS 2000 and NASS/GES 2000 
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2 Rollover Sensors 

The main sensor type that is used in rollover detection systems is the Angular 
Rate Sensor (ARS). These sensors can measure a roll rate, which is defined as 
the change of the angle by time. They are used also in space to measure the 
rotation of satellites or space shuttles and in aeroplanes to help the autopilot 
to work properly. Depending on the precision that is needed different physical 
effects are used in these sensors. In the following the main physical effects are 
explained. 

2.1 Sagnac Effect 

This effect is the most precise, but also most expensive method to measure roll 
rates. It is based on an optical phase shift of light. A laser beam is splitted in 
two beams and coupled anti-parallel into an optical gals fibre wire or interfer- 
ometer. The fibre is winded up in many loops. The phase shift of the beams 
that are combined again after passing the fibre is a function of the roll rate of 
the fibre. In 1913 the French physicist Sagnac did a first experiment with 
some mirrors (figure 5). 



Fig. 5. Principle of Sagnac effect [5]. 

In his experiment the light is splitted by a beam splitter. One part is mirrored 
right hand, the other left hand on a circle and combined again. When the 
whole experiment is rotated the interference pattern is changing. The rela- 
tivistic interpretation of this phenomenon is that the light is passing the circle 
agciinst the direction of rotation, which is moving, in a higher accelerated sys- 
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tem. Therefore it is younger than the light that is passing the circle in the 
direction of rotation. Therefore the phase is different. The Sagnac phase can 
be written as: 



Modern laser-optical rate sensors using this principle can measure rates in 
the range 0.0003 °/s to 400 °/s. The drift is less than l°/h. These sensors are 
quite expensive and are used in the automotive industry only as reference 
sensors in crash tests. Sometimes the sensors are more expensive than the 
whole car. 



2.2 Precision Effect 

These types of sensors use the precision effect acting on mechanical gyro- 
scopes (figure 6). A micro mechanical gyroscope is rotation with the angle 
velocity £2. If the whole system is rotated with the angle velocity Q, a torque 
M is occurring. As a result a force F can be measured. The Force is a meas- 
ure of Q. The precision of these types of sensors is much lower than the opti- 
cal, but they are cheaper. For automotive applications they are still too expen- 
sive, because of the micro-mechanical system that is needed. 





Fig. 6. Precision effect acting on a gyroscope. 
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2.3 Coriolis Effect 



The Coriolis force can be observed in non-inertial systems. It is called also a 
"virtual force". The principle can be seen in figure 7. A piece of mass is rotat- 
ing on a circle at the angle velocity Q. If the radius of the mass is changed, the 
inertia /is also changed. As a consequence a moment Mis generated: 



A/,. = — = 
dt 



Vfij V a V, 

£ 2 — = i2m — r = 



~ O V V 
-2 • m • £2 X V • /■ 



dt 



dt 



The Coriolis Force is given by: 
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V 
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Fig. 7. Coriolis force. 

In sensors the piece of mass is a tuning fork (figure 8). The upper parts of this 
fork are oscillating. Therefore the radius r is changing permanently. If the 
whole tuning fork is rotated with at angular velocity Q, the moment Af is caus- 
ing a change in the direction of oscillation. This additional motion is perpendi- 
cular to the basic oscillation and can be measured if the tuning fork is part of 
a capacitor, which changes the capacitance because of this additional motion. 
This is also a micro-mechanical system, but it is much cheaper than the gyro- 
scope type. The price of such sensors is less than 50 Euro and they are used in 
automotive applications. They can measure roll rates up to 300 °/s by a preci- 
sion of about 1 °/s. Drift of these sensors has to be compensated by software in 
the application. The primary oscillation is in the kHz-range. 
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Fig. 8. Tuning fork principle. 



3 Rollover Detection 

The main sensor in a detection system is the Angular Rate Sensor as written 
above. The signal of sensor is integrated by time to get the predicted rell angle. 
To be able to detect a non zero staring position (initial angle unequal 0°) addi- 
tional sensors are required. For example a z-log-^ sensor can measure the angle 
between the earth gravity field direction and the sensor position. The same 
sensor can be used to compensate a wreng angle calculation because of a pos- 
sible bias. If the measured roll rate of the angular rate sensor is unequal 0° in 
case that there is no physical rate, the integrated angle would increase con- 
tinuously. To avoid this a bias removal logic has to be implemented. 
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Fig. 9. Delphi rollover sensing module. 
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It is also possible to measure the static roll angle by the use of perpendicular 
oriented linear (they measure in linear direction) accelerometers (y-low-g, x- 
low-g). In this case the Angular Rate Sensor can measure changes in the actu- 
al roll angle caused by high roll rates and the linear sensors can measure slow 
changes. Therefore a combination of such sensors result in the best-calculated 
angles. But it has to be considered that accelerations due to high speed curves, 
rough road conditions and misuse steering manoeuvres may occur. This can be 
achieved by useful software solutions. 



4 Arming Concept 

The entire measuring system consisting of sensors, micros and firing loops has 
to be verified by itself as good as possible. The reason is simple: It has to make 
sure that any failure in the system may not result in an unexpected deploy- 
ment of the airbags or other restraint systems. The system is usually checked 
at power on and during normal operation. To improve the security level a sec- 
ondary so called "arming path" is used. A deploy can occur only in case that 
both paths (main discrimination path and second arming path) detect a 
rollover situation. The arming path validates that an intended event is occur- 
ring. The goal is to reduce the number of unexpected deployments due to sin- 
gle-point electrical faults, unrecognised faults of the sensors and undesired 
trigger commands due to electro-magnetic induction. 



Discrimination Path 




Arming Path 



Fig. 10. Arming concept. 
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A simple (cheap) way of realisation can be used as arming path, but it should 
be as independent as possible. In other words the arming path should use 
other sensors, other microchips and a different software algorithm if possible. 
The bigger the independence is the bigger is the safety level. The concept is 
visualised in figure 10. 



5 Calibration 

Each rollover algorithm has to be calibrated. The calibration process is based 
on a set of rollover tests and misuse tests. The rollover tests should ensure that 
the algorithm detects the rollover situation early enough to deploy the 
restraint systems or the roll bars (in case of convertible cars). Normally sever- 
al stages can be detected. For each stage (different roll rates or different max- 
imal roll angles) there should be several rollover tests present for calibration. 
A set of misuse tests should ensure that there is no rollover detected even 
under rough driving conditions. Also airborne scenarios shall not result in an 
deployment if there is no rollover. 

The calibration is tested in different ways. First on PC level a model of the 
whole rollover detection system is tested. All hardware including the toler- 
ances and the software is included in this model. The calibration engineer can 
verify the behaviour of the system. Therefore he uses a library of real rollover, 
crash data and computer generated scenarios. Second the calibration is tested 
in the lab on the real hardware. Therefore the real module is modified on the 
bench. The electrical line between the sensor and the micro is cut and exter- 
nal electrical signals are connected to the input of the micro. Signals generat- 
ed by the PC can simulate signals from the sensors. If the bench can work 
automatically, thousands of crash scenarios and rollover situations can be used 
to verify what the real hardware is doing. The output is also monitored and 
recorded by the PC. 



6 Simulation and Modeling 

Delphi developed several modern rollover concepts and is continuously 
improving them. To do this Delphi applies Physics, Math, Electronics and 
Software Engineering expertise to create auto-code rollover algorithm building 
blocks for next generation algorithms. 
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The software is tested automatically by special other software tools. By the 
help of this, software weaknesses can be found easily. For example it can be 
investigated how the software is reacting upon modification of the sensor sig- 
nals as well as other hardware parameters or hardware tolerances. These 
investigations become more and more important, since it is not easy to find 
bugs in complex software. 




Fig. 11. Simulation tools. 



7 Outlook 

Rollover has become more and more important in automotive industry. 
Especially for convertibles and SUVs this issue is discussed heavily. In the 
United States in the next years there will be some legal regulations expected. 
In the European Community a workshop including automotive suppliers, uni- 
versities, OEMs and ensurance companies are working on rollover classifica- 
tion, modelling, sensing and useful safety strategies. It is possible to have stan- 
dardised rollover tests as ENCAP tests on the market. Today a modern car 
should have a four or better five star rating in this test. Tomorrow the cus- 
tomers will also ask for the safety performance in a rollover condition. Even 
today you can find the first rollover crash tests in some automotive papers (e.g. 
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German ADAC). The knowledge and the technical realisation are present 
today. The customer's request will come tomorrow. 
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Abstract 

This paper describes two newly developed low-cost thermoelectric 
infrared imaging sensors, having a 48 x 32 element thermoelectric 
focal plane array (FPA) and a 120 x 90 element thermoelectric FPA, 
respectively, and two experimental vehicle systems. The FPAs pro- 
vide high responsivity of 2.100 V/W and 3.900 V/W, respectively. 
The FPAs are basically fabricated with a conventional IC process and 
micromachining technologies and have a low cost potential. One 
experimental system incorporated in the Nissan ASV-2 is a blind 
spot pedestrian warning system that employs four infrared imaging 
sensors. This system helps alert the driver to the presence of a 
pedestrian in a blind spot by detecting the infrared radiation emit- 
ted from the person's body. The system can also prevent the vehicle 
from moving in the direction of the pedestrian. The other is a rear- 
view camera system with an infrared detection function. This sys- 
tem consists of a visible camera and infrared sensors, and it helps 
alert the driver to the presence of a pedestrian in a rear blind spot. 
The sensor performance is suitable for use in various safety and 
occupant comfort systems. 



1 Introduction 

Advances in electronics in recent years have led to the adoption of many dif- 
ferent types of sensors on vehicles, including pressure sensors, accelerometers 
and visible charge-coupled device (CCD) cameras. These devices are being 
used to meet societal demands for cleaner exhaust emissions, lower fuel con- 
sumption and improved safety and comfort, among other requirements. In 
Japan, the safety performance of vehicles has been a rising concern in con- 
junction with the increase in traffic accident fatalities since the late 1980s. 

Infrared (IR) radiation refers to a form of light having wavelengths longer than 
those of visible light. Various types of warm bodies, including the human body, 
emit IR radiation or visible light corresponding to their temperature. Infrared 
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sensors capable of detecting such radiated energy are an extremely useful 
means of human body detection because they can detect the presence of a per- 
son even at night without any illumination. 

The IR radiation emitted by the human body, however, is of the long wave- 
length type (LWIR) in the 10 pm wavelength band, and its detection requires 
some ingenuity because of its extremely low energy level compared with that 
of visible light. Detection methods until the 1980s were limited to the use of 
cooled sensors (HgCdTe detectors, etc.) that were cooled to ultra-low tempera- 
tures in the vicinity of 77 K. Such sensors could not be used on ordinary pas- 
senger cars because of such aspects as the cooling system service life, their 
weight and power consumption. 

However, two types of uncooled IR imaging sensor, which had been originally 
developed as a U.S. public sector technology [1,2], were implemented in com- 
mercial applications in 1992. These sensors provide detection performance 
nearly equal to that of cooled devices. The IR focal plane array (IR-FPA) used in 
these uncooled IR imaging sensors requires accurate temperature control in 
the vicinity of the phase transition temperature, necessitating the use of an 
additional device such as a Peltier thermeelectric cooler. Even uncooled IR 
imaging sensors, which have fewer compenents than cooled IR imagers, are 
still high in cest, a factor that limits their automotive use to high-end luxury 
cars at present. To promote widespread use of IR imaging sensors on vehicles 
in the coming years, it is necessary te reduce their cost further. We have 
focused on the thermoelectric type of IR-FPA [1-4] that is highly compatible 
with the conventional IC manufacturing process, does not require any tem- 
perature control mechanism or optical chopper and allows easy design of the 
second-stage processing circuit because of its thermoelectric nature, all of 
which help to give it a lew cost potential. A CCD type of thermoelectric IR-FPA 
with over 10.000 pixels has already been announced [5]. However, we adopt- 
ed a CMOS imager system [6,7] with the aim of reducing the cost further, and 
have attained high sensor performance by miniaturizing and optimizing the 
thermeelectric IR-FPA. 

Moreover, we have used the results of our thermoelectric IR imaging sensor 
research to develop a prototype nighttime pedestrian warning system [8] and 
a blind spot pedestrian warning system. These systems were developed 
through our participation in the first and second phases of the Advanced 
Safety Vehicle (ASV) project prometed by the Ministry of Transport (currently 
the Ministry of Land, Infrastructure and Transport) beginning from 1991. 
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This paper describes the thermoelectric IR imaging sensors, the blind spot 
pedestrian warning system incorporated in the Nissan ASV-2 and a rear-view 
camera system with an IR detection function. 



2 Infrared Imaging Sensor 

2.1 Infrared Radiation Emitted by Human Body 

Infrared radiation is a form of electromagnetic radiation having wavelengths 
from 0.78 to 1.000 pm and is one type of light wave. With respect to IR appli- 
cations, the bandwidths having high transmittance in the atmosphere (atmos- 
pheric window) are divided into the short wavelength IR (SWIR) region, mid- 
dle wavelength IR (MWIR) region and long wavelength IR (LWIR) region, as 
shown in figure 1 . Selective use is made of these different wavelength regions 
depending on the target temperature. 




Wavelength (pm) 

Fig. 1. Spectral transmittance characteristics in the atmosphere. 

The wavelength characteristics of the IR radiation emitted by the human body 
are determined by its radiant exitance, which is determined by the absolute 
temperature of the human body surface and its emissivity. As the first step of 
this study, the atmospheric temperature dependence of the facial surface tem- 
perature and the influence of solar radiation were measured in an environ- 
mental testing facility for automobiles. Under a shady condition, with only illu- 
mination and air-conditioning ventilation, the facial surface temperature 
showed a very small change of 4.1 °C in relation to an atmospheric tempera- 
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ture change of 30 °C, as shown in figure 2. Even under a condition of mid- 
summer solar radiation (height of the illumination used for solar radiation was 
3 m and the energy density at the floor was 2.76 MJ/m^ h), it was found that 
the facial surface temperature rose only 1°C compared with that under the 
shady condition. At an atmospheric temperature of 27°C (300 K) or lower, it 
was observed that a temperature difference of 5°C or more was obtained with 
respect to the facial surface temperature. 




10 15 20 25 .30 .35 40 45 

Atmospheric temp. (deg. C) 



Fig. 2. Dependence of facial surface temperature on atmospheric temper- 
ature and solar radiation (height of the face: 1.7 m; air-conditioning 
ventilation only; height of illumination used for solar radiation: 
3 m; solar energy density: 2.76 MJ/m^ • h). 



The characteristics of IR radiation emitted by the human body were then 
determined. An object that absorbs all light is referred to as a blackbody, and 
it also becomes a complete radiator under the application of Kirchhoff's law. 
The emissivity of the human body is around 0.98, which is close to that of a 
blackbody. The spectral radiant exitance of a blackbody is determined by 
Planck's radiation law and is given by 






2jthc" 

-l) 



( 1 ) 



where is the emitted radiation per unit wavelength and unit area 
(W/cm^ • pm), h is Planck's constant (= 6.63- W-s^), k is the Boltzmann 

constant (= 1.38 • lO'^^ W- s/K) and 7’ is the absolute temperature (K). The max- 
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imum radiation intensity wavelength can be found with Wien's displace- 
ment law as 



A^r=2897|.im-K 



( 2 ) 



As is clear from figure 2, the facial surface temperature is nearly constant in 
a range of 31-36°C (304-309 K). Therefore, the maximum radiation intensity 
wavelength of the IR radiation emitted by the human body is =< 9.5 pm, as 
indicated in figure 3. Accordingly, use of LWIR radiation in a wavelength range 
of 8-13 pm is suitable for human body detection. 




Wavelength (p m) 

Fig. 3. Spectral radiant exitance of a blackbody. 



2.2 48 X 32 Element Thermoelectric IR Sensor 

The thermoelectric IR-FPA consists of tiny thermocouples connected in series 
that utilize the Seebeck effect to convert the temperature difference between 
the hot and cold junctions into a voltage. In order to increase the temperature 
difference, the thermocouples are formed on a thin membrane that is fabri- 
cated in a micromachining process. The thermocouples are fabricated of p- 
type and n-type polysilicon layers, consisting of materials commonly used in 
general semiconductor processes and connected in series. 
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Figure 4 shows the functional principle of the sensor element. Incident IR radi- 
ation is absorbed by the Au-black absorber in the center of the sensor and con- 
verted to thermal energy which is transmitted in turn via the hot junctions, 
beam and cold junctions by means of heat radiation to the Si substrate that 
serves as a heat sink. To increase the thermal resistance between the hot and 
cold junctions, a portion of the Si substrate is removed to form a cavity. In addi- 
tion, high sensitivity is attained by using a vacuum-sealed package to avoid 
internal heat conduction through the air. The responsivity R of the thermo- 
electric IR-FPA is given by 



R^naR„n 



(3) 



where n is the number of thermopile pairs, a is the Seebeck coefficient (sum of 
p-type and n-type), R^fj is the thermal resistance between the hot and cold 
junctions and t| is infrared absorptivity. Equation (3) indicates that increasing 
r) and R^fj is effective in improving responsivity R. The optimum values of r| 
and Rf/j were calculated on the basis of the sensor size and the accuracy of the 
fabrication process. Additionally, a precisely patterned, high-IR absorptivity 
Au-black layer, which was developed independently in our laboratories, was 
adopted to improve t]. As a result, absorptivity of more than 90% has been 
achieved around a wavelength of 10 pm. A scanning electron microscope 
(SEM) micrograph of one element is shown in figure 5. 

This pixel structure was then used to fabricate a prototype 48 x 32 element 
thermoelectric IR-FPA. A micrograph of the 48 x 32 element IR-FPA chip is 
shown in figure 6. The dimensions of the various parts of the sensor are as fol- 
lows: chip size of 10.5 x 7.44 mm^, pixel pitch of 190 x 190 pm^ and ther- 
mopile width of 0.8 pm for the six pairs of thermopiles. The positive terminal 
of each pixel is connected to an output line through two built-in NMOS tran- 
sistors controlled by an external input signal. The negative terminal of all the 
pixels is connected to the same common terminal. Additionally, because this 
device has comparatively high internal resistance of 116 kW, it is necessary to 
limit the bandwidth of the second-stage circuit for the purpose of reducing 
Johnson noise (thermal noise). In order to obtciin the desired video frame rate, 
the entire sensor is divided into 1 2 blocks of four columns each, and signals are 
output in parallel from 12 output lines. This vacuum-sealed IR-FPA achieves 
responsivity of R = 2,100 V/W and a thermal time constant of 25 ms, repre- 
senting high levels of performance compared with similar commercial devices. 
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Fig. 4. Functional principle of the sensor element. 



Au-black 
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Fig. 5. A SEM micrograph of one element of the 48 x 32 element thermo- 
electric IR-FPA. 

This thermoelectric IR-FPA with its low cost potential was then incorporated 
into an IR imaging sensor that has been developed independently in our labo- 
ratories for automotive applications. A photograph of the IR imaging sensor is 
shown in figure 7, and its specifications are given in table 1. The IR imaging 
sensor is a compact, lightweight device, measuring 100 mm in width, 60 mm 
in height (excluding the bracket) and 80 mm in depth and weighing 400 g. The 
sensor has a germanium meniscus single-element lens with a focal length of 
f = 15 mm and ^0.7. Roth sides of the lens are coated with an anti-reflection 
coating having a wavelength of 10 |tm. This IR imaging sensor operates under 
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a program stored in a Read Only Memory (ROM) incorporated in the Central 
Processing Unit (CPU, SH7034). The 2-D array sensor outputs successive sig- 
nals in parallel from the 12 blocks based on an address signal request from the 
CPU. The output signals are amplified and sent to the CPU via an 8-bit analog- 
to-digital converter (ADC). The CPU performs offset compensation and respon- 
sivity compensation and then makes a pedestrian detection judgment. A func- 
tion is also included for converting video data to an NTSC video signal by 
means of a video digital-to-analog converter (DAC). This IR imaging sensor can 
output a pedestrian detection signal and image data in the NTSC video signal 
format via the RS-232C interface. With its ^0.7 lens, the IR imaging sensor 
achieves a Noise Equivalent Temperature Deviation (NETD) of less than 0.4°C. 




Fig. 6. Micrograph of the 48 x 32 element thermoelectric IR-FPA (chip 
size: 10 x 7.44 mm^). 

Figure 8 shows a facial image of a person wearing eye glasses that was 
obtciined with this IR imaging sensor. The eyeglasses portion appears as a dark 
region because infrared rays in the 10 pm wavelength band do not pass 
through glass. 




Fig. 7. 



Automotive IR imaging sensor incorporating the 48 x 32 element 
thermoelectric IR-FPA. 
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1 Periormancc Paratnclcr 


Capability (@.tOOK) I 


Array configuration 


48 X 32 elements 


Chip dimensions 


10.5 mm X 7.44 mm 


Pixel size 


190 pm 


Spectral response 


8 - 13 pm 


Responsivity 


2100 V/W 


Thermal time constant 


25 ms 


Lens 


f = 15 mm, f/0,7 


NETD (@ F0.7) 


0.4°C 


Field of view 


30° X 20° 


Outputs 


RS-232C, Detection, NTSC 


Power 


12 V. 0.25 A 


Overall dimensions 


1 00 mm X 60 mm X 80 mm 


Weight 


400 g 



Tab. 1 . Specifications of the IR imaging sensor. 




Fig. 8. IR image obtained with the automotive IR imaging sensor. 



2.3 1 20 X 90 Element Thermoelectric IR Sensor 

A prototype 120 x 90 element FPA with a 100 pm element pitch was devel- 
oped with the aim of increasing the resolution and detection range. The ele- 
ment pitch was narrowed from 190 pm to 100 pm to reduce the overall size of 
the chip. To maintain the desired performance despite the smaller element 
pitch, thermal resistance 7?,^ was increased by reducing the beam width from 
14 pm to 4.4 pm. A micrograph of the prototype FPA is shown in figure 9. The 
overall size of the chip is 14.4 mm x 11.0 mm. The imaging area is 
12.0 mm X 9.0 mm because the element pitch is 100 pm. In order to reduce 
the scanning frequency and Johnson noise, the device is divided into twelve 
blocks (Block #1 -Block ^12) for every ten columns, similar to the 48 x 32 ele- 
ment FPA, and is scanned in parallel. In other words, this device operates as 
twelve 900-element FPAs arranged in parallel. Each element consists of two 
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pciirs of a p-n polysilicon thermocouple and an NMOS transistor for Y-address 
selection. The measured electrical resistance of the prototype sensor was 
90 kQ. An FPA was mounted in a vacuum-sealed package with a Ge window 
and responsivity was measured by using infrared radiation emitted from a 
blackbody furnace at 500 K. The average responsivity of all 10.800 elements, 
however, was 3.900 VAA/, which agreed well with the calculated results and is 
higher than any previous value reported for thermoelectric FPAs. A prototype 
IR imaging sensor incorporating the 120 x 90 element thermoelectric IR-FPA 
is shown in figure 10. The IR imaging sensor consists of two units. A head unit 
equipped with the IR-FPA and a Ge lens measures 136 mm in width, 112 mm 
in height (excluding the bracket) and 35 mm in depth. The Ge lens is a menis- 
cus single-element lens with a focal length of f = 15 mm and f/0.7. Infrared 
images taken with the FPA are shown in figure 1 1 . These images are shown 
with offset and responsivity compensation. 




Fig. 9. Micrographs of a 120 x 90 element focal plane array. 




Fig. 10. A prototype IR imaging sensor incorporating the 120 x 90 element 
thermoelectric IR-FPA. 
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Fig. 11. Infrared images taken with a 120 x 90 element focal plane array 



3 Blind Spot Pedestration Detection System Research 
3.1 Concept 

This system was incorporated in the Nissan ASV-2. The concept of this system 
is to detect infrared radiation emitted by the human body or other objects with 
the aim of helping to prevent blind spot accidents by alerting the driver to be 
careful and by preventing the vehicle from moving in the direction of a detect- 
ed person. This is accomplished by using four IR imaging sensors, mounted at 
the front and rear, which detect children or other heat sources in the driver's 
blind spots (within 3 m and within the field of view of the sensors) where they 
cannot be seen and could become obstacles as the vehicle begins to move. The 
concept of the system is illustrated in figure 12. 

TWo new functions were added to this system, taking into account the charac- 
teristics of blind spot accidents. One is a human body detection function using 
four IR imaging sensors, and the other is a function for preventing the vehicle 
from moving by means of braking control. There can be a time difference 
ranging from several seconds to several minutes between the time a driver 
enters a vehicle and starts the engine and the time the vehicle is put in motion. 
It is possible that the circumstances around the vehicle may change during 
that interval. Because children in particular are apt to do unexpected things, 
incidents have been reported where blind spot accidents occurred even though 
the driver confirmed the safety of the environment around the vehicle before 
getting in. The IR imaging sensors, capable of detecting IR radiation emitted 
by the human body regardless of whether it is day or night, were adopted for 
this system because it was thought that a function for selectively detecting the 
human body and other heat sources would be the best way of helping to pre- 
vent such blind spot accidents. 
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The function for preventing vehicle movement by braking control was added 
because there are instances when the extremely short distance between a 
vehicle and an obstacle does not allow sufficient time for the driver to execute 
an evasive maneuver. 



Only rearward movement is allowed IDC 



c> 







Only forward movement is allowed 








Temperature difference between 
pedestrian md background AT > 5®C 

Fig. 12. Concept of the blind spot pedestrian warning system. 



3.2 System Configuration 

The system configuration is shown in figure 13. Infrared imaging sensors 
mounted at the vehicle's four corners serve to detect a pedestrian in the dri- 
ver's blind spots. In the event a pedestrian is present in the direction in which 
the vehicle is about to move, the system issues audible and visual warnings to 
alert the driver and activates throttle and brake actuators to prevent the vehi- 
cle from moving. The components making up the system include: four IR 
imaging sensors incorporating a pedestrian detection judgment capability; a 
blind spot pedestrian warning control unit that integrates the signals from the 
IR imaging sensors and sends a signal to the Control Area Network (CAN); a 
vehicle motion control unit or an Auto Hox (main controller) that manages vehi- 
cle motions; an integrated Human-Machine Interface (HMI) control unit; a 
monitor and a warning unit; and throttle and brake actuators that prevent the 
vehicle from starting off. 



Low-Cost Infrared Imaging Sensors for Automotive Applications I 75 



Main C/U (Aulo liox) 

IR imaging sensor IR imaging sensor 

Vehiele 




Inlegraled Ml II C U Blind spol pedeslrian warning CAJ 



Fig. 13. Overview of the blind spot pedestrian warning system. 



3.3 Detection Range of IR Imaging Sensors and Mounting Positions 

The mounting positions of the IR imaging sensors are closely related to their 
detection range. A study was made of the mounting positions under the fol- 
lowing conditions. 

► IWo IR imaging sensors each would be used at the front and rear, for a 
total of four sensors. 

► The sensors would have a field of view of 30°. (horizontal) by 20°. (ver- 
tical). 

► The detection range would be 3 m under the condition that the tem- 
perature difference AT’ = T^- = 5°C. 

► The sensors would be mounted so that they did not interfere with the 
lamps, cooling equipment, tailpipe and other vehicle parts. 

An experiment was conducted to measure the potential blind spot areas 
around the Nissan Cima (FY33) sedan that was the base vehicle of the Nissan 
ASV-2. The eye point in the driver's seat was set at a height of 1.05 m from 
the ground. Measurements were made of the areas where the ground (ground 
height of 0 m) could not be seen directly with the eye or by means of the mir- 
rors from the driver's seat, assuming that the task was to detect a human body 
lying on the ground. The results are shown in figure 14. It was found that the 
blind spots at the front were within a distance of 2.7-4 m from the vehicle 
body and those at the rear were within a distance of 9-11 m. Tkking those 
results into account, the positions of the four IR imaging sensors were deter- 
mined so that they satisfied the four conditions mentioned above. 
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Photographs of the test vehicle are shown in figure 15. In figure 15h, two IR 
imaging sensors of the same type as those used at the front are mounted 
inside a germanium window at the rear. As indicated in figure 15a and 15h, 
the two front sensors were installed at a height of 370 mm and were spaced 
1.580 mm apart, and the two rear sensors were mounted at a height of 
515 mm and were spaced 1.000 mm apart. The sensors at hoth the front and 
rear were positioned so that their optical axis was horizontal. The front sen- 
sors were installed at a relatively low height on account of the shorter blind 
spot distance at the front, while those at the rear were mounted higher 
because of the longer blind spot distance in the rearward direction and also to 
avoid interference with the tailpipe. The front sensors were spaced farther 
apart because a larger detection range was needed at the front than at the 
rear in connection with the front-wheel steering of the vehicle. The front and 
rear detection ranges thus determined for the system are indicated in figure 
14b. With the present system, there are small areas very close to the front and 
rear of the vehicle where detection is not provided. To cover these areas as 
well, either the field of view would have to be expanded or approximately 
three sensors each would have to be installed at both the front and rear. 




Fig. 14. Road surface areas that cannot be seen from the driver's seat of the 
Nissan ASV-2 base vehicle (Cima) at an eye point height of 1.05 m 
from the ground (left) and detection areas at AT = 5“C (right). 



Low-Cost Infrared Imaging Sensors for Automotive Applications I 77 



3.4 System Operation 

The flowchart in figure 16 outlines the operational sequence of the system. At 
the time the vehicle is stopped, it is assumed that there are no pedestrians in 
the front and rear detection ranges. In response to a request signal from the 
Auto Box, the IR imaging sensors capture baseline image data (image A) and 
record the data in their internal memory units. When a vehicle is parked for a 
long time, it is presumed that the atmospheric temperature and sunlight will 
change, so the baseline video data (image A) are updated accordingly. The IR 
imaging sensors take images at periodic intervals of approximately 30 s and 
update the baseline image data (image A) if there is no heat source in the 
image. The baseline image data are not updated when a heat source is pres- 
ent. 




Fig. 15. Nissan ASV-2 equipped with the blind spot pedestrian warning sys- 
tem. (a) Front view, (b) rear view (with Ge window) and (c) enlarged 
front view. 



Also, at the time the vehicle is about to start off, the IR imaging sensors cap- 
ture start-off image data (image B) in response to a request signal sent from 
the Auto Box. To remove the influence of background heat sources, a differen- 
tial image (image C) is calculated by subtracting image A from image B. The 
IR imaging sensors then process differential image C to detect heat sources. 
When a heat source is detected, a detection signal and the position of the sen- 
sor that detected the heat source (e.g. right front) are sent to the Auto Box via 
the blind spot pedestrian warning control unit. If the position of the sensor 
that detected the heat source (i.e. front or rear) coincides with the vehicle's 
intended direction of movement, the Auto Box judges that a pedestrian is pres- 
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ent in that direction. It then alerts the driver by means of an audible warning 
and a display screen indicator, using signals sent through the integrated HMI 
control unit. 



Vehicle is stopped 

T ~ 

Aeqiiisition of ^ 

baseline image data 



Acquisition of images 
for iipd^iiie use 




Creation of differential 
image 

C = y-Aj 



Processing for detection of heat sources 




Prevention of vehicle 
movement, 
audible wanting, 
display screen indicator 



Fig. 16. Operational sequence of the blind spot pedestrian warning system. 

The Auto Box also activates brake and throttle actuators that forcibly prevent 
the vehicle from moving in the direction of the detected pedestrian. 
Conversely, if the sensor position and direction of movement do not coincide, 
the Auto Box judges that there is no pedestrian in the vehicle's direction of 
movement and allows the vehicle to proceed. 
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The system judges that a heat source is present if an object corresponding to 
a temperature difference of 4°C or greater is in 10 or more pixels of the above- 
mentioned differential image C. That number of pixels is approximately equal 
to the size of a human face at a distance of 3 m (corresponding to a 1 5 cm 
angle). 



3.5 Detection Results 

Human body detection distances were measured under different atmospheric 
temperature conditions to verify the operation of the prototype system. The 
results are given in figure 17. In a low temperature range of 10-2 2 °C, the 
detection distance was 6-8 m, which would cover a large portion of the rear- 
ward blind spot shown in figure 11. On the other hand, the detection distance 
decreased to 3-3.5 m in a temperature range of 24-30°C. Under a condition of 
a temperature difference tsT = (surface temperature of a pedestrian) - 
(background temperature) >5°C (i.e., the atmospheric temperature (back- 
ground temperature) is no more than 27°C), a detection distance of 3 m or 
greater was obtained with the prototype system specifications. Additionally, it 
was also confirmed in actual vehicle tests conducted with the Nissan ASV-2 
that the system detected the presence of a human body and prevented the 
vehicle from moving in that direction. These results provide confirmation that 
the prototype system is effective in helping to protect pedestrians in potential 
blind spots. 
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Fig. 17. Relationship between IR imager detection range and atmospheric 
temperature. 
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4 Rearview Camera with IR Imaging Sensors 

This system is a fusion of a visible rear-view camera and the IR imaging sen- 
sors described above, enabling it to alert the driver to the presence of pedes- 
trians in the rearward blind spot. The in-vehicle configuration of the system is 
shown in figure 18, and the positions of the devices are shown in figure 19. 

An image processor combines the image signal from the visible camera with 
the detection results of two IR imaging sensors to produce a rearview image 
that is shown on a dashboard display screen. As shown in figure 18, the IR 
detection area achieved with the two IR imaging sensors is a trapezoid that 
measures 1.7 m along the width of the vehicle), 2.7 m on its long side and 
2.1 m in distance from the vehicle. The long-side dimension with the IR imag- 
ing sensors and the distance along the ground are 2.7 m. A detected heat 
source is indicated on the screen by a red dot at the center of gravity of the 
detected region as shown in figure 20, and an audible warning is also given to 
alert the driver. 

Based on the results shown in figure 17, the system is capable of detecting a 
human body in the entire detection area under a condition of 




Fig. 18. System configuration. 
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IR image sensors 



-Visible camera 



Fig. 19. IR camera system installed on Nissan ASV-2. (a, left) Enlarged view 
of installation positions and (b, right) entire rear-end. 





Fig. 20. Display screen showing detected human body. 



5 other IR Sensing Applications 

Occupant detection systems for a smart air conditioner, an intruder alarm, a 
advanced air bag technology (AABT) have been developing. Prototype systems 
are equipped on the experimental vehicle illustrated in figure 18. Another 
system for recognizing hand commands is also being developed. 






6 Summary 



The characteristics of IR radiation emitted by the human body were investi- 
gated by measuring the dependence of the facial surface temperature on the 
atmospheric temperature and solar radiation. The results made it clear that 
the facial surface temperature shows little change under a condition of a light 
wind. 

Thermoelectric IR imaging sensors with 48 x 32 elements and 120 x 90 ele- 
ments, respectively, were developed that have a low cost potential. The 
48 X 32 element sensor was used to develop a prototype blind spot pedestrian 
warning system that was incorporated in the Nissan ASV-2. In addition, it was 
also used to develop a prototype rearview camera system with IR detection 
capability. 

The human body detection range is influenced by the background tempera- 
ture and is 3 m under a condition of a temperature difference AT = 5°C. This 
detection range may be effective for detecting the presence of pedestrians in 
potential blind spots around a vehicle, however, further research work in this 
area is needed. 

In future research work, it will be necessary to enhance the responsivity of the 
IR imaging sensor so that it functions more effectively under various environ- 
mental conditions and to improve the weatherability of the optical system. 




Low-Cost Infrared Imaging Sensors for Automotive Applications I 83 



References 

[1] C. Hanson, "Uncooled thermal imaging at Texas Instruments", SPIE, Vol. 1689, pp. 
330-339, 1993. 

[2] R. A. Wood, C. J. Han, and P. W. Kruse, "Integrated uncooled IR detector imaging 
arrays". Proceedings of the IEEE Solid State Sensor and Actuator Workshop, pp. 
132-135, 1992. 

[3] JJ.Yon, E. Mottin, L. Biancardini, L. Letellier, and JL. Tissot; "Infrared 
Microholometer Sensors and Their Application in Automotive Safety", Advanced 
Microsystems for Automotive Applications 2003, Springer Verlag, ISBN 3-540- 
00597-8, pp 137-157. 

[4] T. Ishikawa, M. Ueno, K. Endo, Y. Nakaki, H. Hata, T. Sone, M. Kimata, T. Ozeki, 
"Low-cost 320 X 240 uncooled IRFPA using conventional silicon IC process", SPIE, 
Vol.3698, pp.556-564, 1999. 

[5] T. Kanno and M. Saga, "Uncooled infrared focal plane array having 128 x 128 
thermopile detector elements", SPIE, Vol. 2269, 1994. 

[6] M. Hirota and S. Morita, "Infrared sensor with precisely patterned Au-black 
absorption layer", SPIE, Vol. 3436, pp. 623-634, 1998. 

[7] M. Hirota, Y. Nakajima, M. Saito, F. Satou, and M. Uchiyama, "120 x 90 element 
thermopile array fabricated with CMOS technology", SPIE, Vol. 4820, pp. 239- 
249, 2002. 

[8] M. Hirota, M. Saito, S. Morita, and H. Fukuhara, "Nighttime Pedestrian 
Monitoring System and Thermal Infrared Technology", Jidosha Gijutsu, Vol. 50, 
No. 11, pp. 58-63, November 1996 (in Japanese). 

Masaki Hirota, Yasushi Nakajima, Masanori Saito, Makoto Uchiyama 

Nissan Motor Co., Ltd. 

1 Natsushima-cho, Yokosuka 

Kanagawa 237-8523 

Japan 

m-hirota@mail.nissan.co.jp 

Keywords: infrared, sensor, thermopile, automotive, blind spot, pedestrian detec- 

tion, occupant detection 




Application of Micro Scanning Laser Doppler Vibrometer for 
Vibrational Velocity and Resonance Frequency Measurement 
on Micro Electro Mechanical Devices (ME MS) 



85 



P. Castellini, B. Marchetti, E. P. Tomasini, Universita Politecnica delle Marche 



Abstract 

Micro-Electro-Mechanical Systems (MEMS) represent the integra- 
tion of mechanical elements, sensors, actuators, and electronics on a 
common silicon substrate through the utilization of microfabrication 
technology. The "micromachining" processes selectively etch away 
parts of the silicon wafer or add new structural layers to form the 
mechanical and electromechanical devices. MEMS promises to revo- 
lutionize nearly every product category by bringing together silicon- 
based microelectronics with micromachining technology, thereby, 
making possible the realization of complete systems-on-a-chip. 
MEMS is truly an enabling technology allowing the development of 
smart products by augmenting the computational ability of micro- 
electronics with the perception and control capabilities of microsen- 
sors and microactuators. MEMS is also an extremely diverse and fer- 
tile technology, both in the applications it is expected to be used, as 
well as in how the devices are designed and manufactured. MEMS 
technology makes possible the integration of microelectronics with 
active perception and control functions, thereby, greatly expanding 
the design and application space. 

Nevertheless, the spread of the micro-electro mechanical systems 
(MEMS) market is strictly related to the development of manufac- 
turing technologies and to the implementation of analysis and diag- 
nostic technique that allow to increase performances and reliability. 

In this paper we demonstrate the feasibility of the laser doppler 
vibrometer, which has been designed and developed for measuring 
in the microscale, to characterize the dynamic of MEMS, in terms of 
measuring the vibration velocity in several point of the sample and 
revealing the resonance frequencies of the structure. 




1 Introduction 



In a previous work [2], the design and development of a workstation based on 
a laser Doppler vibrometer has been presented. In this paper we will demon- 
strate the application of such measurement technique on MEMS devices in 
order to study their vibrational behaviour 

Microelectromecanical systems (MEMS) refer to devices that have a charac- 
teristic length of less than 1 mm but more than 1 pm, that combine electrical 
and mechanical components and that are fabricated using integrated circuits 
batch-processing technologies. 

MEMS are finding increased applications in a variety of industrial and medical 
fields. The automotive Industry uses MEMS pressure sensors to measure 
engine oil pressure, vacuum pressure, fuel injection pressure, transmission 
fluid pressure, ABS line pressure, tire pressure, and stored air bag pressures. 
MEMS accelerometers can be used to trigger the air bag or lock seat belts. 
MEMS temperature sensors can be used to monitor oil, antifreeze, and air tem- 
peratures. Resonant transducers are an example of a MEMS sensor that have 
been attracting a widespread scientific and technological interest in the devel- 
opment of miniaturized chemical and biological sensors. They can be used for 
the measurement of pressure, acceleration, strain, temperature, vibration, 
rotation, proximity, acoustic emission, and many others. 

In order to characterize this wide range of different devices the laser doppler 
vibrometry is a valuable technique thanks to his non-contact nature and to the 
possibility to position the laser beam in every point on the object surface. 

With this instrument it is possible to dynamically characterize the systems: to 
find resonance frequencies, distinguish mode shapes, revealing the existence 
of all the vibrational modes, nonlinear behaviour and also to investigate the 
mechanism of mechanical energy dissipation that play a fundamental role in 
the performance of the devices (quality factor assessment) [3, 4]. 



2 Measurement System 

The system is based on a commercial Laser Doppler vibrometer, in which the 
optical set-up, the mechanical arrangement and the processing software and 
hardware were modified and developed to measure vibrations with a resolu- 
tion in the micro scale. 
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The mmn characteristics of the developed system is a very versatile platform, 
in which laser doppler vibrometry, digital signal processing and image acqui- 
sition and processing can work together (figure 1). 




Fig. 1. Scheme of the measurement system. 

In order to scan the sample surface, it is necessary to position the laser beam 
on the points of a measurement grid built over the sample surface (figure 3), 
we used two different approaches: 

► To move the object by means of mechanical micropositioner, maintain- 
ing the laser beam in a fixed position. 

► To move the laser beam across the surface of the MEMS by means of 
piezoelectric mirrors with the sample in the same position (figure 2). 

A fundamental element of our measurement system is represented by the 
focusing objective that allows us to reach a suitable spatial resolution by reduc- 
ing the laser spot dimensions. The objective that we implemented is composed 
by two lenses with a focal length of 30. 1 7 mm corrected for the spherical aber- 
ration that reduce the laser spot dimensions allowing a special resolution of 
5 pm (laser spot diameter). 
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Fig. 2. Scheme of the measurement system with piezoelectric mirrors to 
move the laser heam. 



Another important element of our system is the post-processing software that 
was developed in LahVIEW environment. The main characteristics required 
were the ability to integrate the control of each component (stages, cameras, 
laser etc) and to make easier and automatic the measurement procedure. 

In particular it had to answer to the need of: 

► Managing the images acquisition. 

► Making possible to build the measurement grid. 

► Managing the excitation system. 

► Managing the acquisition parameters. 

► Managing the data post-processing. 

► Managing the measurement procedures: 

► Vibrational analysis. 

► Automatic Q factor calculation. 
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4 Vibrational Analysis 

The measurements were performed on out of plane accelerometers. The sam- 
ple was excited by using white noise to determine the resonance frequencies 
and with sinusoidal signal at the identified resonance frequencies. 




Fig. 3. Scheme of the out-of-plane accelerometer. 



With our system it is possible to obtain the time history, the frequency spec- 
tra, and the maps of the vibration velocity and the displacement. 

In figure 4 is shown the average spectrum acquired over the measurement 
points of the grid by exciting the sample with white noise. It is possible to 
notice the presence of a resonance peak at the frequency of about 35 kHz and 
hits second harmonic at the frequency of 70 kHz. 
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Frequency spectrum 




Fig. 4. Frequency spectrum. Excitation signal: white noise. 

In figure 5 is shown the time history obtained hy averaging the data acquired 
over the 25 points of the measurements grid. The sample was excited with a 
sinusoidal signal at the frequency of 35 kHz. Each measurement point was 
acquired with a number of averages varying from 4 to 32 depending on the 
noise level. 
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Fig. 5. Time history. Excitation signal: sinusoidal wave. 
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In figure 6 is shown the measurement grid that has heen built over the sam- 
ple image by the developed software. The laser beam is positioned over each 
measurement point by the piezoelectric mirrors driven by the same software. 




Fig. 6. Measurement grid. 

With the microvibrometer it was possible to measure the vibration velocity 
and, by post-processing the data to obtain the vibration velocity map and the 
displacement over the accelerometer surface as shown in figure 7 and 8. It is 
possible to notice that the higher velocity and displacement are, as expected, 
in correspondence of the centre of the accelerometer. 
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Fig. 7. Vibration velocity map. 




Fig. 8. Displacement map superimposed on the microaccelerometer. 
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7 Conclusion 

In this paper has been demonstrated the LDV system is a very suitable tool for 
measurement in the microscale thanks to his non-contact nature and to the 
possibility to position the laser beam in every point on the object surface. 

With this technique it is possible to dynamically characterize the systems: to 
find resonance frequencies, distinguish mode shapes, revealing the existence 
of nonlinear behaviour and to calculate the Quality factor of the devices. 

In this research work we applied such technology to study the dynamic behav- 
iour of out-of-plane microaccelerometers. We were able to determine the reso- 
nance frequency of 35 kHz and to reconstruct the map of the vibrational veloc- 
ity and of the device displacement. The visualization of the displacement is 
very useful to verify that the device is moving accordingly to the design spec- 
ification and detect the presence of anomalies in the MEMS behaviour. 

It is also possible to superimpose the obtained map with the picture of the 
sample acquired by the CCD camera to have an easier visualization of the 
obtciined results. 
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Abstract 

Visual sensors are going to play an important role in driver assis- 
tance systems. The authors present examples for the application of 
visual sensors. They present an overall architecture for a general 
technological platform and individually selected aspects. Special 
emphasis is attached to systematic testing and evaluating. They 
present a test procedure with virtually generated driving scenes. 



1 Visual Sensors in Driver Assistance Systems 

1.1 The Role of Visual Sensors in Driver Assistance Systems 

Important traffic signs have signalling colours, reflecting coatings of road 
marking increase safety, and streets are illuminated by spotlights and street- 
lamps during night. These are only some examples showing that traffic infra- 
structure is designed and optimised for recognition by the human eye. That is 
why cost-effective driver assistance systems are necessarily going to be 
dependent on using such signal sources in an adequate way. In addition, the 
risk of mutual influence is reduced in mass application by waiving active com- 
ponents (e.g. radar), and another cost benefit is achieved. Thus, visual sensors 
are going to play an important role in future driver assistance systems. 

Intensive research in the field of image processing opens perspectives for a new 
handling of vehicles by man. The vehicle acquires with its sensor technology 
knowledge about the environment, reprocesses it, links it intelligently, and 
presents it to the driver in an appropriate way. Due to its great number of pos- 
sible applications and technical variants of realisation, it is a great challenge to 
make an essence out of this mass of theoretical ground work and to develop a 
product ready for the market out of this. For this, you have to make the step 
from theory to practice and to manage the tremendous difference between pro- 
totypes and mass production. What does this cost? How stable is it? There are 
many questions emerging on the way, which have to be answered. 
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1.2 Application Examples of Visual Sensors 

We would like to show the application of visual sensors in driver assistance sys- 
tems with the help of four examples. 

At lane departure warning (see figure 1), images of a camera system turned to 
the front are analysed in order to generate a model from the lane ahead by 
visual analysis of the road marking. Out of this, the system generates estima- 
tion about the lane intended to drive within the next seconds and minutes. 
Simultaneously, the actual driven lane is always carried along by analysing the 
vehicle's CAN data. A warning signal can be produced by continuously com- 
paring such data in order to draw the driver's attention to an unintentional 
deviation from the lane. 




Fig. 1. Lane departure warning. 

The included lane identification functionality is a requirement for many other 
applications, like e.g. the automatic triggering of a braking process when 
obstacles appear on the lane, the alignment of the front lights to the course of 
the lane, or a semi-automatic support when driving bends. 




Fig. 2 



Traffic sign recognition. 
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Ti’affic sign recognition (see figure 2) has a sensor arrangement very much 
related to the above mentioned one. A camera turned to the front is used here 
too, in order to recognize traffic signs located near the lane. The recognized 
traffic signs are saved. In this way, the current applicable speed limit can be 
signalised to the driver by analysing the speed limit signs. The recognition of 
stop signs or give way signs can help compensating the driver's inattention 
and observing the give way rules applicable at the next junction. 

Unlike the above said, blind spot detection (see figure 3) requires a camera for 
this, which can be placed into the side mirror, and which is focused on the 
blind angle. The system permanently analyses the motion dynamics in this 
area and fades in a warning light in the rear mirror as long as there is an object 
in the blind angle. In addition, the calculated motion dynamics can help to 
warn of overtaking vehicles approaching at very high speed. 




Fig. 3. Blind spot detection. 



The principle applied for this (expanding a man's field of vision by a camera) 
is used for the backing camera too (see figure 4). A wide-angle camera mount- 
ed at the rear is focused on the rear bumper's area. 
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It releases the driver from the agony increased by more and more rounder and 
higher rear contours to back either like flying blind using the bumper's 
crushability or wasting space for shunting. 




Fig. 4. Rear view system. 



During this process, the wide-angle lens' distortion is corrected by the soft- 
ware automatically. Subsidiary lines faded in show the driver valuable distance 
information and support the parking process. Depending on the current steer- 
ing wheel's turning angle the tyres' track, which would occur if mmntaining 
this turning angle, is shown additionally. 



1.3 Questions Raised during the Development of Visual Sensors 

The entry of visual sensors in the vehicle is connected with some questions, to 
some of which there are currently no definite answers. It will be decided in the 
coming years, what answers and which technology will succeed in mass appli- 
cation. 

If you just look at one single of the above applications at first, you will find 
already a profusion of technical variants when choosing an appropriate camera 
system. A binocular system with two lenses is able to extract better depth infor- 
mation from the scene cempared with a monocular system and thus, it facilitates 
three-dimensional modelling for the following algorithms. This advantage is 
faced with higher costs and a more complicated assembly. Apart from the opti- 
cal features (like e.g. the aperture angle), it is also important when selecting the 
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camera whether a colour camera is used or grey scale values are sufficient. 
Requirements to the sensor's night view capability as well eis general prescrip- 
tions from the automotive sector are increasing the selection conditions further. 

If several applications of the above kind shall interact, it will automatically 
raise the question of common utilization of (here in particular the cameras') 
resources. With how many cameras will a future car be equipped? How many 
cameras will be required for an all-round view? At what places will they be 
installed? How does this go together with the designer's ideas? What is the 
enduser's acceptance regarding various realisation variants? 

These open questions are already reason to believe that the integration of visu- 
al sensors will have great dynamics in the future and that short innovation 
cycles are to be expected. These considerations have a direct impact on the 
development of visual sensors. For using components evolving from this in dif- 
ferent application scenarios, they should be highly reusable. In addition, the 
principles of rapid prototyping will be useful in order to realize various alter- 
natives within short time and to develop those very promising further. 
Development environments, which permit a systematic evaluation, will be par- 
ticularly important for this. 



2 General Framework for Embedded Visual Algorithms 

The positioning of suppliers of visual sensors within the automotive industry's 
gear is in the state of flux. Suppliers are faced with the question how intensive 
they should concern themselves with this new technology. You cannot assign 
visual sensors unambiguously to just one automobile component. You may 
combine them with headlights and integrate them into bumpers and rear mir- 
rors. In the interest of reliable and compatible systems, it is wise to generate 
technical platforms for visual sensors, which can be used in many ways, and 
which can be integrated into different environments. 

Moreover, such a technological platform makes a common utilization of 
resources possible. You may use the same camera for various tasks, and stor- 
age and computing performance can be concentrated on applications running 
in the foreground. 

Visual sensors are a new issue for many suppliers. We would welcome if it 
couid be possible to avoid proprietary isolated applications from the very 
beginning and to develop common interfaces similar to the aims of the 
AUTOSAR initiative. 
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We outline the architecture of development platform visual sensors below and 
present reusable components. 



2.1 Architecture of a System for Rapid Prototyping 

A modular software design is natural. A standard regarding the exchanged 
data should exist for the separate modules to be able to communicate with 
each other. This concerns both the semantic level (which terms and objects are 
distinguished) and the concrete representation on the implementation level 
(data format). The principle way of exchanging data is part of the design deci- 
sions to be made early. In this case, we suggest the data flow orientated 
approach, at which three types of components exist: 

► Data sources produce data and send them along the determined vari- 
able data flow. 

► Data processing stations receive data from different sources, modify 
and forward them. 

► Data sinks receive data and display them usually to the user or save the 
data. 

We recommend a bi-directional approach in general, at which the data flow 
can be actuated from data sources when sensor data are present, as well as 
into the opposite direction by requesting data through data sinks, if an output 
device has to be operated at a constant data rate for example. 

A system's overall architecture for rapid prototyping with visual sensors can 
look like the one below: 

You can differentiate between four essential components. In the sensor layer, 
the user may select from existing sensors by Plug&Play. These sensors can be 
switched on/off individually, and they produce a data stream of independent 
sensor data at first. This data stream is then bundled by time stamps in a syn- 
chronized way and is made available to the second layer. This layer too is put 
together by Plug&Play from a database of components compiled beforehand. 
Additional connections between these processing modules may be determined 
here individually. The separate connections are associated with data types, like 
e.g. CAN data, image data, or intermediate data of calculations. 

The third layer serves to output the results, usually displayed on a monitor. 
Alternatively, generated data may be fed back into the CAN bus. You may con- 
figure this layer too as you like it by Plug&Play. 
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A fourth component is working in the background. You can record with it both 
sensor data and the results, in order to reproduce the same situation later and 
if necessary compare the results of new algorithms with those of older config- 
urations. 

As we have already mentioned before, you can already see at this schematic 
level that you have to determine the data types for communication between 
the separate components. 
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Fig. 5. Software architecture for rapid prototyping of visual sensor appli- 
cations. 



2.2 Examples of Reusable Software Components 

Let us take individual components as an example, which are playing a role in 
the development of visual sensors. 

Currently available cameras usually have a maximum bit depth of 12 bits, 
which permit 4.096 different brightness values. Using logarithmic character- 
istic lines permits a great scope of dynamics, but at the expense of the resolu- 
tion between the discrete brightness values. That is why; it is necessary to use 
the characteristic line optional. In most cases, you have to adjust the camera 
additionally, i.e. to adjust the camera's set-up parameters (like e.g. the expo- 
sure time) to the respective situation. Since this adjustment can be done only 
based on the camera images present beforehand, which are dependent on the 
set-up parameters' history, this problem is a question of a control circuit. You 
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can think of various solutions, which also have their relevant justification, for 
the requirements on the camera's control vary depending on the application. 
If the image is shown to the user directly there should he no brightness oscil- 
lations, but the system should react as fast as possible to the change in bright- 
ness. However, if on the other hand images are becoming part of the image 
processing algorithms, it might be necessary to adjust as slowly and smooth as 
possible, so that extreme jumps in brightness do not occur between the single 
images. In this case, the algorithms could get problems in the follow-up of rec- 
ognized features. 

Another algorithm is the Bayer pattern reconstruction in colour cameras with 
Bayer Pattern, at which RGB values are calculated from the mosaic like spread 
colour information for each pixel. Especially when implemented in embedded 
systems some algorithms are eliminated because of their increased computing 
time. In addition, using colour cameras implicates another range of algorithms 
to get a satisfactory colour quality like e.g. an automatic white balance. 

A library of basic algorithms should be provided on the mathematical level. 
This covers the range from simple matrix operations, via homography up to 
the implementation of complex processes, if problems arise with auto calibra- 
tion for example. The aim is to carry out a camera self-calibration based on just 
one frame rate, i.e. to determine the image parameters, which represent their 
optical features on the one hand as well as their position relative to the vehi- 
cle on the other hand. 

Statistical methods are used too. A hundred percent safe procedures are sel- 
dom in image processing. Image recognition is often connected with a certain 
probability. However, in order to reach reliable results you can work with two 
supporting methods. On the one hand, the multiparadigm approach suggests 
itself, at which at least two different processes are working on the same input 
data and are synchronizing their results afterwards. On the other hand, 
hypotheses are often used, i.e. the algorithms establish hypotheses at first, to 
which probabilities are already being allocated. Lists of hypotheses are gener- 
ated by combining evaluations of single images, which can be analysed statis- 
tically in order to extract the most probable hypotheses. 

All these problems can be isolated and looked at as a black box with specified 
in- and outputs. If necessary, several solutions can then be provided for each 
problem. 
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2.3 Software Development Structure 

The research process in the field of visual sensors is rapidly moving forward. 
New ideas are swirling at its front tamed and bundled in the maelstrom of 
research. Stabilized procedures establish themselves in the rear area and 
become the status quo. Software has a similar structure. New developments 
are changing quickly are revised, and some of them are taken over into the 
core. The core code has stabilized by having reused it, and a proven structure 
has formed in the course of time. 

Classes like e.g. image format, image data, CAN message, mathematical meth- 
ods, models (matrices, homographies, transformations, ...), and much more are 
part of this structure. You cannot measure the value of such classes in code 
lines, because exactly that selection process is essential, which took place 
before and where alternatives have been tested and turned down. It is impor- 
tant to be aware of the fact that the code is only a small part of that knowl- 
edge, and that the essential part is the ideas, experiences, and concepts behind 
it. 

You can design the core in such a way that it may be used both for prototypes 
(however, you need an additional layer for the user interface for this) and for 
its implementation in embedded systems. It is recommended working with 
prototypes and using their short development cycles. 

A requirement engineering tool, like e.g. DOORS, IRqA, CaliberRM, or Xtie is 
wise for taking down the requirements and target definitions. A performance 
specification is better structured by systematic collection, and some questions 
are answered before the implementation cycle starts. The time you invest here 
pays to sped up implementation. 



2.4 Common Hardware Components 

When selecting the hardware architecture for visual sensors it is of crucial 
importance that you have to reach a very high data throughput and that some 
algorithms require an additional buffering of single images. That is why the 
internal storage has to be dimensioned sufficiently and the access to external 
storages has to be fast enough. The camera, the CAN bus, the video output, 
and if necessary individual signals have to be supported for user interaction, 
because they are the essential in- and output interfaces. 



Manufacturers (like e.g. National Semiconductor Inc.) offer promising solutions 
in the field of CMOS sensors, which have already built in algorithms to 
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improve image quality or for image compression. When integrating a camera, 
we suggest the Firewire interface for prototypes or distributed systems, which 
is suitable for automotive according to the IDE- 1394 standard, and which in 
particular makes a synchronous triggering of several cameras over one single 
connection possible. 




Fig. 6. Aglaia camera components. 

When selecting a camera, we suggest using CMOSs cameras for the following 
reasons among others: 

► CMOS technology is becoming a mass product increasingly with a cor- 
responding impact on pricing. 

► CMOS sensors have high dynamics and permit the definition of image 
sections as a region of interest. 

► CMOS technology is easy to handle and facilitates circuit integration. 

It is advantageous for the integration of CAN data to include an intermediate 
level converting manufacturer specific CAN messages into generally usable 
symbolic designators. 



3 Tests and Evaluation 

Unlike research orientated prototype development, you should allow reliabili- 
ty and robustness a central role in order to market visual sensors successfully. 
Here, a special feature of image processing algorithms comes into it for they 
often have a heuristic approach in their core, so that you can establish the 
results' quality first after having carried out numerous series of tests even at 
an absolutely perfect implementation. 



Mobile Vision - Developing and Testing of Visual Sensors for Driver Assistance Systems I 105 



The reliability of recognizing the lane is depending on the weather and on the 
light conditions for example. Reflections on the lane can make recognition 
more difficult. Road marking may be interrupted at junctions or building site 
areas. You can face this challenge in different ways. We would like to present 
two components here. 



3.1 Quality Assurance through Environment Models with OpenGL 

Using records of everyday driving scenes for the evaluation of systems with 
visual sensors, the problem of an unambiguous collection of the target results 
arises. If traffic signs shall be recognized for example, the actual traffic signs 
appearing on the records have to be collected in an appropriate way at first. 
According to the current state of the art, you can do this in a reliable way man- 
ually only, by which an evaluation becomes very cost-intensive. Using other 
data sources (e.g. laser aided object recognition or from maps with GPS data) 
is limited too. 

You avoid those disadvantages with the following simulation test method. For 
this, an artificial 3D-model is generated at first. In this process, you can select 
various elements from a library, like vehicles and their movements, lanes, traf- 
fic signs, trees, etc. for example. Their positions and shapes are parameterised. 
Those parameters (e.g. the position and type of the respective traffic signs) do 
directly correspond to the target values. This 3D-model is represented by 
OpenGL and is presented to the image-processing algorithm in such a way, as 
if those data are coming from the sensor directly. You can reach a realistic rep- 
resentation through further effects like textures, shadow casting, mist, and 
much more (see figure 7 and figure 8). 




Fig. 7. OpenGL simulation for blind spot application. 
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By this approach, the values established by the algorithm can be compared 
directly with the target values and be evaluated statistically reliable. This 
method is completely reproducible, so that even different algorithm can com- 
pete against each other. 




curve ok»jecb vehxcXe subject vehicle 

redius » 4082m v » 115)cfn/h v ^ 901on/li 

distance = 15m yaw ra'te = 0*/s 



Fig. 8. OpenGL simulation for lane detection. 

A particular advantage of this method is that it can be completely atomised, 
and that you can quickly build up a database as large as one likes by modifi- 
cation of the parameter sets. 



3.3 Record and Playback of Driving Scenes 

The above-described method allows a very systematic approach. However, it 
includes the disadvantage that the examined (abstract generated) driving sit- 
uations are unable to reproduce reality completely. That is why they are sup- 
plemented by tests, in which the initial data are records of realistic scenes 
including typical cases and extreme situations. Such tests are carried out dur- 
ing the whole development process and are accompanied by additional field 
tests, in which a great number of test vehicles are used for adjusting and val- 
idating the results. 

In this process, it is important to record the sensor data occurring during a test 
drive in such a way that a later repetition in the laboratory is as much as pos- 
sible equivalent to the original situation. In addition to the task recording each 
sensor separately, such a recording system is faced with particularly high 
requirements regarding the data-recording rate and with the data volume 
resulting from this. You should particularly focus on the sensor data's syn- 
chronous playback for the sensor signals' temporal relation to each other has 
a strong influence on the algorithms. 
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Driving scenes with data from the cameras, the vehicles (CAN), and the cur- 
rent calibration etc. are collected via such a recording module and transferred 
to a server. Such data collection is a requirement for quality checks and evalu- 
ations carried out continuously. 



3.3 Conclusion 

The data flow orientated development environment Cassandra® of Aglaia 
GmbH for Rapid Prototyping with a stabilized core of image processing algo- 
rithms and data structures and the MobileScope module for synchronous 
recording and playing back of sensor data is a good basis for a economical 
development of systems with visual sensors and their product scaling to 
embedded systems. An essential foundation stone for automatic evaluation is 
laid by the test component for artificial creation of driving scenes. Further 
hardware components for a general technical platform for visual systems are 
currently prepared and may be used in various fields of application. 
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Abstract 

The sensor industry has seen a drastical change due to replacement 
of mechanical structures with MEMS technology making sensors 
smaller and more cost effective [1, 2], With higher integration, 
shrinking size and automated processes the quality and robustness 
was enhanced dramatically in the same time to meet high volume 
automotive applications demands. This process can be seen as an 
ideal synergy due to the high volume production that leverages the 
high capital and fixed costs. The lessons learned from this roadmap 
was brought back to a simple looking device - a capacitive absolute 
pressure sensing element as bare die. This article will describe the 
way how this element was shrunk and optimised for the next gen- 
eration of low power Tire Pressure Monitoring and Systems without 
battery. 



1 Evolution of Capacitive Pressure Sensor Elements Due to TPMS 
Requirements 

A group of engineers started in Finland in 1980 a development of pressure 
sensors targeted for weather measurement application. Today their work is 
found in meteorological radiosondes manufactured in high volume and in high 
accuracy barometric instruments. After 20 years of using this sensor technol- 
ogy in highly sophisticated high accuracy systems the same technology was 
applied in a low cost, mass producible, robust, low power pressure sensor for 
an extremely harsh environment in automotive tires. 

In a cutting edge product like battery-less TPMS critical design parameters are 
involved. In the application the following factors were identified: 

► Size: The entire system needs to be extremely small since it should be 
mounted in the tire in several applications. Tire tags have been pre- 
sented with the size of a 1-cent coin. 

► Costs: The added value perceived by such systems by the end user is 
relatively small. Since legal regulations in the U.S. will most probably 
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lead to a 100% equipment rate and 4 to 5 measurement units are 
required for one car the price per unit is constantly under surveillance. 

► System mounting and integration: For the less weight (and therefore 
small size) demand it is necessary to optimize each component. The 
weight of the TPMS unit in the wheel needs to he compensated since it 
is positioned unsymmetrical in the wheel leading to dynamic distur- 
bances. 

► Robustness against overpressure and temperature stability: One 

option is that the sensor in the system will he vulcanized into the tire - 
proof pressure requirements of more then 32 bar are seen as require- 
ment as well as exposure to a very high temperature, all this without a 
change in calibration or reliability. 

► Centrifugal acceleration: The measurement shouldn't be effected by 
centrifugal acceleration or shocks from the road up to 1000 g. 

► Media compatibility: The tire is sealed so that there isn't typically any 
exchange of air. Outgasing of several more or less harmful agents could 
lead to severe damage of electronic - one is sulfur. 

► Stability over lifetime: Over the specified lifetime of 10 or 15 year a 
deterioration of the sensor functionality isn't acceptable. 

► Low power consumption: In battery application the charge is limited 
and should guarantee a lifetime of more than 10 years. Battery-less sys- 
tems are even more restrictive since the entire power needs to be trans- 
mitted by induction to the tire module and form factors of antenna and 
their positioning are pretty much dependent on the necessary energy 
that needs to be transferred so that this variant is defined as ultra low 
power application. 

► Signal ratio: Due to the low power consumption requirement signal 
conditioning becomes critical. A comparatively large sensor element 
signal has the advantage to be much better signal conditioning includ- 
ing less vulnerability against EMC. 

► Variation and temperature behavior: Small part to part variation and 
excellent over temperature behavior reduces the effort that needs to be 
taken into energy budget for signal conditioning. 



2 Concept to Manufacturing: Small Improvements in All Details 

Starting with the most important parameter for the entire sensor element 
design several simulation were made to define the best size which was still ful- 
filling the requirements of all other functions. It turned out that a footprint of 
1,4 X 1,4 mm2 brought the best optimisation. Smaller size would have had the 
disadvantage in the system manufacturing that the part couldn't be handled 
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anymore. The height was determined to be 1.0 mm as a good compromise. 
Thinner devices wouldn't have had the required stiffness to compensate 
mechanical stress that would have effected the measurement accuracy. 
Thicker elements would have meant bigger dicing grooves which would have 
reduced the number of elements per wafer. 




Fig. 1. Image of VTIs APS4. 

In the simulations determining the size the most important factor was to keep 
a high dynamic measurement range. Hence the small size this range was kept 
as high as 7 pF for 0 bar to 14 pF for full scale pressure. This was achieved by 
an extremely small distance between the two capacitor plates of nominal 
1 pm. The passive parallel capacitance was kept small by using a thick glass 
layer between the thick and thin silicon layer with silicon feedthrough; this 
provides a good active to passive capacitance ratio. The transfer function fol- 
lows the equation: 



C = C -r 

'-(X) ^ 




( 1 ) 



whereas Cq is large compared to ^00- The realised values are in the range of: 
^00 = 1 pF and Cq= 5 pF. Figure 2 is showing measurement results of first pro- 
totypes, an 8 bar element. 
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Reflecting the over temperature uniformity of the transfer curve provides the 
perspective that a temperature compensation wouldn't he necessary for this 
type of element. The major two reasons for this positive behaviour are the 
above mentioned small Cqo impact as well as the fact that silicon is used as a 
conductor only so that the temperature dependency of other semiconductor 
properties isn't effecting the over temperature behaviour. This advantage 
needs to be preserved by using appropriate signal conditioning and mounting 
technologies which should avoid thermal related stress to the sensor element. 




Fig. 2. Transfer function of APS4 of first prototypes (measurement 
results). 

Thckling the long-term stability the anodic glass bonding needs to seal for life- 
time. Therefore good process controls in production are required. VTI has 
gciined experience in glass bonding technology for close to 20 years in series 
production. Glass is therefore a well known material but has also some specif- 
ic characteristics like outgasing and absorption of residual gas on the glass sur- 
face which would lead to change of the internal pressure. This effect was 
addressed by adding wet etched gas pockets connected to the reference pres- 
sure volume enable good long-term stability. The difficulty in the given design 
was to find any space in the optimised element structure. The positioning of 
that vacuum reservoir is shown in figure 3 schematically and in figure 4 as a 
cross section in silicon. 

A major application driven design rule was that the connecting pads should be 
on the lower side of the element so that the part can be directly mounted to a 
carrier like a RGB or ceramic. Another advantage is that the size of the element 
is reduced by the pad size that was consuming space on the surface and was 
difficult to protect against aggressive media. Figure 5 shows older pressure 
sensor designs, APS2 and APS3, which had bonding pads on the upper side. 



Silicon Capacitive Absolute Pressure Sensor Elements for Battery-less and Low Power TPM I 113 



The design of APS4 with two different terminals on the lower side was 
realised by glass isolator in the middle of the element. The glass is also struc- 
tured in a way that the offset capacity value is kept as small as possible. 
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Fig. 3. Schematic cross section APS4. 

The diaphragm thickness is determined by the full-scale pressure range. The 
design is dimensioned in a way that sensor elements from 1 bar (20 pm) to 
30 bar (80 pm) would be realised. By using wet etching the height of the 
diaphragm can be processed quite accurate. The gap is always kept at 1 pm to 
remain with the same dynamic range for all the potential variants of this con- 
cept. The diaphragm thickness is determined by the full-scale pressure range. 
The design is dimensioned in a way that sensor elements from 1 bar (20 pm) 
to 30 bar (80 pm) would be realised. By using wet etching the height of the 
diaphragm can be processed quite accurate. The gap is always kept at 1 pm to 
remain with the same dynamic range for all the potential variants of this con- 
cept. 




Fig. 4. Vacuum reservoir for long term stability (left); diaphragm thick- 
ness and distance between capacitor plates (right). 



Underlying the design of the diaphragm was the requirement to avoid any sig- 
nal changes caused by centrifugal force as well as mechanical shock caused by 
rough road surfaces. For that reason the boss shown in figure 5a was removed 
scarifying some linearity of the transfer function. 
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Further shape optimisation of the diaphragm herder area brought a good com- 
promise for the transfer function. 




Fig. 5. APS2 with boss (left); APS3 with bonded contact areas (right). 



3 Signal Conditioning 

The sensor behaviour is very good in line with the parallel plate model accord- 
ing to the formula: 



= ( 2 ) 

Figure 6 shows that there is only a small deviation between the transfer curve 
of first sensor prototypes and the ideal parallel plate model. The maximum 
deviation can be read as approximately 7 mbar. 
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Fig. 6. Small difference between parallel plate model and sensor output. 
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With capacitive and piezoresistive measurement principles there are compet- 
ing concepts on the market. Whereas the piezoresistive principle is in today's 
TPMS systems in series production it didn't meet the requirement of battery- 
less systems anymore due to the fact that the power consumption is much 
higher and for a realisation sleep modes for the measurement acquisition is 
mandatory. The power consumption for the APS4 sensor element is quite low 
since the parallel resistance to the capacity higher than 10 G£2 leading to very 
low leakage currents. 

The signal conditioning can be done by several concepts whereas the typical 
switched capacitor technology is extremely accurate and robust against dis- 
turbances but consumes too much power to be an appropriate approach for 
this ultra low power application. The good over temperature stability of APS4 
pressure sensor element helps to avoid a reference capacitor in element tech- 
nology. The sensor capacity can be connected directly to the oscillator circuit- 
ry of the ASIC interface. 



4 Implementing APS4 into Appiication Structures 

The vertical groove glass process enables a MEMS component to be a reflow 
solderable SMD component. This leads to easy positioning of the sensor ele- 
ment with other SMD components so that wire bonding isn't necessary as 
interconnection technology at all (figure 7). 

The media compatibility can be realised by using an appropriate sealing mate- 
rial that covers the element up to the glass passivation so that only the glass 
passivation and the silicon diaphragm are exposed directly to the harsh envi- 
ronment. This gives an advantage compared to other structures which are 
exposed on the surface of the element. 




Fig. 7. Proposal for mounting of APS4 on PCB or ceramic. 
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The sidewalls of the sensor below the glass passivation (electrodes shown in 
yellow) have to be protected since foreign media with large dielectric constant 
could change the parasitic capacitance of the element on the bottom edge of 
the sensor element sidewalls - the minimum gel height is shown as a red line. 
While using a potting material for this protection a potential side effect could 
be that the material has a tendency to creep on top of the element due to the 
surface tension. This would imply more mass on the diaphragm which would 
make the sensor more vulnerable for centrifugal forces or shocks like the boss 
design. This can be avoided by the creepage stop, a small groove on all four 
sides of the sensor. The rest of the electronic can be positioned under the pro- 
tective potting also. 

Without having potting material on the surface of the diaphragm the acceler- 
ation sensitivity is in a reasonable level that can be neglected according to: 

- Ps, ■ K. • « 



where as a is the acceleration, is the diaphragm thickness [20. ..80 pm], 
and Pgi is the density of silicon. 

The design was very much optimised for small size and high dynamic range 
so that there is one parameter that needs to be contemplated while mounting 
the device to the application structure as well as for the treatment in the logis- 
tic chain. Due to the small capacitor plate distance of 1 pm the maximum 
acceptable BSD voltage level is less or equal than 100 V. This fact makes spe- 
cial requirements in the mounting area necessary like for MOSFET structures. 
After mounting and encapsulation this special precaution isn't necessary any- 
more in application. 

The sensor element was tested in a set-up where vulcanisation conditioned 
were applied to the sensor element. The conditions were 185°C for 1 h under 
32 bar pressure. The functionality and calibration of the sensor remained 
unchanged after this test. 



5 Trends in TPMS Application 

The benefit of having the correct tire pressure regarding optimal braking dis- 
tance, less tire wear out and unexpected tire break down was not only shown 
in several papers [3, 4] but also proven by a series of accidents driving this 
topic to public awareness. Therefore the market is demanding the introduction 
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of TPMS for the following application classes: Passenger cars need a full scale 
pressure range of 5 bar; light trucks are defined for 8 bar whereas truck would 
need sensors with full scale pressure values of 12 bar to 13 bar. Elements for 
all three pressure ranges will be available from VTI. 

Indirect pressure measurement as shown in [5] was one of the attempts to 
cover unexpected pressure loss in one of the wheels hence it seems to have the 
disadvantage of being unable to identify equal conditions of too low pressure 
in all four wheels. Therefore it is most likely that direct pressure measurement 
in all four wheels will become a standard. Based on the minimum requirement 
for measuring the actual pressure in all wheels several additional information 
is driving the development of more sophisticated systems. The tire position 
identification is definitively a basic demand also since otherwise different 
pressures for front and rear tires can not be handled. The information can be 
used beneath under-pressure warnings also for the adaptation of other sys- 
tems like suspension and braking. 

One trend shows that the tire makers will gain more and more responsibility 
for their products. The most significant option is the traceability for legal pur- 
pose which would require a fixed pair of tire and intelligent sensor that could 
be realised by molding of the pressure measurement unit into the tire. This 
leads to a battery-less system since molding of batteries into tires would be dif- 
ficult to realise. A package with low and high frequency data transmission 
integrated with MEMS sensor provides the working solution without battery. 
Silicon based pressure sensor elements with capacitive principle are key com- 
ponents when these type of systems are designed due to their lowest power 
consumption. 



5 Other Applications 

Robust sensor elements are required by other application also where media 
compatibility is significant. VTI has the capability to expand the shown con- 
cept to derivate products with a full scale pressure range between 1 bar and 
30 bar. There are several application specific possibilities to package a simple 
sensor structure as it was shown in this paper. The signal conditioning and 
communication depends in the same way as the packaging on the application. 
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Automotive Applications for the ALASCA Laser Scanner 
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Abstract 

Driver assistance systems provide road users with enhanced con- 
venience and safety. Sensors for detecting and assessing traffic situ- 
ations are readily available to the automotive end-customer. These 
sensors, however, support first generation applications. In this paper 
an advanced sensor technology together with its applications is pre- 
sented. The Automotive LAser SCAnner ALASCA® supports applica- 
tions such as Automatic Emergency Brake (AEB), Stop and Go, 
Precrash, Parking Assistance, Pedestrian Detection and close range 
monitoring for commercial vehicles (e.g. "Turning Assist"). These 
applications are based on the high efficiency of detection and track- 
ing of the objects in the proximity of the car before a collision occurs. 

Furthermore, ALASCA® has the potential to support the ambitious 
plans of the car manufacturers regarding automatic braking and 
steering of vehicles and accident-free driving in road traffic. 



1 Introduction 

As a result of the evaluation of road accident statistics, special interest is 
focused on distance RADAR/LIDAR alert and emergency brake systems. 
Advanced driver assistance systems (ADAS) have the potential to reduce the 
number of accidents by monitoring the driver and the vehicle as well as the 
vehicle surroundings. The need for the development of ADAS functions is 
backed by the 9/2002 EU initiative "eSafety" which includes an enhanced 
action plan to enforce the implementation of new technologies for active safe- 
ty systems and advanced driver-assistance systems. 

For the reduction of the number of fatalities and injuries in public traffic, it is 
necessary to introduce active safety applications to assist the driver in dan- 
gerous driving situations. The long-term objective of full collision avoidance is 
still a vision at the horizon of future developments. Some ADAS products avail- 
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able in the near future are governed by the philosophy to assist the driver in 
situations when the crash is imminent. 

These collision mitigation applications will help to reduce the consequences of 
crashes to all collision partners. An important prerequisite for such systems is 
a sensor for object detection and tracking, which has a high resolution for the 
localization of the colliding obstacle, a precise tracking capability and a high 
degree of reliability under all driving conditions. 

Hella KG and IBEO AS formed an industrial partnership with the intention to 
develop the ALASCA® laser scanner, which serves the requirements of auto- 
motive applications. In the following the applications served by the ALASCA® 
will be described in detail. Special attention will be paid to the ACC Stop & Go 
and Automatic Emergency Brake (AEB) applications. 



2 Technical Principals of ALASCA 

Prior to starting the operational development of the ALASCA®, the project 
partners thoroughly discussed the interdependencies between the automotive 
applications, their functional requirements, and the laser scanner specifica- 
tions. The result was that the applications "Automatic emergency brake", 
"Stop & Go", "Precrash", "Pedestrian Protection", "Parking Assistant" and 
"Turning Assist" could all be supported by the following set of specifications: 



Measurement range R 

Measurement range on 5% reflectivity targets 


0,5 - 80 ni 
30 111 


Distance measurement error a 


0.05 m 


Scan frequency 


10 to 40 Hz 


Angular horizontal resolution agoR 
Scan cutgle 


0.25° to 1.0° 
max. 240° 


No. of scan planes (vertical layers) 


4 


Vertical field of view overt 


3.2° 


No. of targets per shot 


2 


Laser class 


1 (eye safe) 



Tab. 1 . Technical data of ALASCA®. 



As measurement principle of the laser range finder the "time-of-flight"- 
method was chosen. Here the travel time between the sensor and the target 
provides a measure for the distance of the target based on the fact that the 
speed of light is a constant. 
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Requirements such as the reliable detection of objects over of a wide horizon- 
tal angle and accurate longitudinal detection of the position are warranted by 
the specifications and the measurement principle. 

In order to illustrate the way the specifications correspond to the applications 
the following example may be considered. A typical speed in an urban envi- 
ronment can be set to v^p = 50 km/h. A dark target (i.e. pedestrian leg, black 
clothes) with a width of 13 cm and a reflectivity of 5% can be fully detected 
at R^o/„ = 30 m distance, owing to the accurate horizontal resolution of 
ahoy = 0,25°. The available response time, which corresponds to the velocity 
Vyyp is thus determined to be around 0.05 s to 0.06 s and is sufficiently large 
to activate safety measures. 

During vertical pitching of the vehicle the ALASCA® is able to hold contact to 
the tracked objects because of the 4 scan-layers with a total angle of view of 
a^oyf = 3.2°. Driving under rainy conditions or when the laser scanner window 
is contaminated by dust, are compensated by the ability of the opto-electronic 
detection scheme, which allows for the evaluation of two targets per laser shot. 
While the first "hit" on the laser path may be a raindrop, the second mean- 
ingful target stays on the tracked object. 

The ALASCA® is fully integrated into the body of the test vehicle. For evalua- 
tion purposes 2 integration positions were chosen; vehicle front left/right, and 
front centre. It has to be mentioned that the integration of the scanner into the 
vehicle reduces the available horizontal scan angle. Depending on the desired 
application the integration position must be chosen respectively and some 
applications might require 2 laser scanners (i.e. front left and front right on 
vehicle) to ensure the full angular vision. 

Classification of objects is necessary for many future applications and useful 
for pedestrian recognition. 

ALASCA® classifies four groups of road user by a special algorithm: 

► Trucks. 

► Cars. 

► Bicycles / Motorcycles. 

► Pedestrians. 
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3 Automatic Emergency Braking (AEB) 

3.1 Fundamentals 

AEB is one of the ADAS applications that have been in the focus of the devel- 
opment within the current cooperation between HELLA and IBEO AS. Road 
accident statistics determined that about 50% of injuries caused to car pas- 
sengers take place in head-on collision situations and about 40% of the drivers 
involved in these collisions do not brake at all. These figures reveal the poten- 
tial of the AEB. 



3.2 Functionality 

The AEB function realized within the current project follows a straightforward 
strategy, which is laid out in figure 1. If there is an obstacle in the driving path, 
correct information about the location of the corners of the obstacle is neces- 
sary to calculate possible escape routes. At the point when the driver cannot 
avoid the collision anymore, the emergency braking is automatically activated. 
In the next moment the crash occurs. However, the consequences of the crash 
are greatly reduced compared to the situation where no or weak breaking is 
induced by the driver. 




If the collision is 
unavoidable by 
braking or 
steering, the 
braking system 
will be activated. 




Rsciipe Route 
R = f(vi;) 

Steering 

Distance 



Fig. 1. Schematic of the AEB situation. The obstacle is appearing in the 
vehicle's driving path (left). Contact with the obstacle cannot be 
avoided when the driving path associated with the minimum steer- 
ing radius overlaps with the contour of the obstacle (right). 
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The system is on stand-by by starting the car. The AEB status is shown to the 
driver, like the airbag status. The driver does not take further notice of this 
system whilst driving, until an unavoidable accident occurs. On the occasion of 
an AEB event the driver will be informed by the HMI. 




Fig. 2. Data traces from an experimental AEB situation as function of time. 

Note, that the vehicle velocity appears in negative numbers, start- 
ing at 32 km/h and ending at 12 km/h on contact with the obstacle. 

The AEB activation curve changes from 0 to 1 at 1.5 s, the point in 
time where the accident becomes imminent. 

Figure 2 shows a set of experimental data taken during an AEB situation com- 
parable to the one shown in figure 1 . as a function of time. The road condition 
was dry and clean asphalt. The obstacle consisted of an arrangement of foam 
rubber blocks. Vehicle velocity, obstacle width, distance to the obstacle, and 
activation point of the AEB function are displayed. It can be seen that while 
the measured obstacle width remains nearly constant until the crash occurs 
(point B), the velocity (for reasons of convenience during programming shown 
in negative numbers) is strongly reduced after the activation of AEB as a step 
function (point A). The delay of 0.4 s compared to the electronic trigger set-off 
at 1.5 s results from the available electronic braking system, which had been 
developed for ACC and not optimised for AEB. Nonetheless a significant reduc- 
tion of velocity from 32 km/h to 12 km/h is induced, which translates into a 
reduction of motion energy down to 14%. 



3.3 Requirements on Brake Systems 



The test vehicle used during this project is equipped with a standard series 
can bus system connecting all data sources and processing units. (Note: As an 
exception to this the ALASCA® transmits its data via a real time data bus to 
the processing unit on which the object recognition algorithm is implement- 
ed.) The brake booster consists of a modified ACC booster system. These two 
facts leave space for improvements in the future. While the reaction time of 
the driver can hardly be reduced, the deployment of a high power brake boost- 
er combined with a data bus system, which permits real time activation, will 
be very beneficial in further enhancing the effect of AEB. The next 
HELLA/IBEO AS test vehicle will be equipped with these features, which are 
also a requirement for future series production vehicles equipped with AEB. 



4 ACC Stop & Go 

4.1 Fundamentals 

The application Stop & Go (S&G) is an enhanced function of ACC (Adaptive 
Cruise Control) for special situations like traffic jams. Like ACC it is a comfort 
function to reduce the stress of the driver but to let him have full responsibil- 
ity for the vehicle. 

ACC Systems are realized on RADAR and LIDAR basis. They are well adapted 
for far distance applications. For example the Hella ACC System IDIS® has a 
range of 150 m with a horizontal aperture angle of 16°. The detection area of 
an ACC System is different from an S&G system. S&G systems are well adapt- 
ed for low speed and short-range targets. Therefore, it is the logical addition to 
the ACC function. For S&G, short and middle range distances in front of the 
vehicle are sensed with a large horizontal aperture angle (see figure 3). Also a 
high accuracy of the distance measurement data is necessary in cases of driv- 
ing through small passages. Cutting-in of objects have to be recognized. These 
features are supported by the fusion of the ALASCA® and the IDIS® systems. 
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Fig. 3. Overlapping field-of-vision areas of ALASCA® and IDIS®. 



Extended requirements for the application ACC S&G compared to ACC only: 



Large horizontal aperture angle 


120° up to 150“ 


Fine distance measurement resolution 


1 cm 


Fine horizontal angular resolution 


0.25° up to 1° 


System availability for velocities 


0 km/ll up to 1 50 km/ll and above 


Considering of stationary obstacles 



Advanced deceleration values 

Tab. 2. Extended requirements for ACC S&G 

The current sensors for ACC, designed for long-range detection, and ACC S&G, 
designed for the short and medium range, cover all possible velocity ranges in 
the case of sensor fusion. For the sensor fusion, managing of the different 
modes of the respective sensors is necessary. 



4.2 Driver Control 

The driver is able to activate or deactivate the system. By activating the sys- 
tem he can select a desired speed and a desired following distance to the rel- 
evant object ahead. The ego-vehicle follows the relevant object ahead with a 
constant distance and stops automatically behind the relevant object if neces- 
sary. For a driver initiated or automatic restart ALASCA® scans the proximity 
due to its precise distance and angular resolution. 
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As the S&G SYStem acts only with moderate deceleration, it is necessary to 
inform the driver via a Human Machine Interface (HMI) that he has to take 
over control of the vehicle via brake intervention in cases where collisions can- 
not be avoided. All set points for (1) selected speed and (2) selected following 
distance as well as (3) a detected relevant object will be displayed to the driv- 
er. Overdriving without deactivation can be carried out by the gas pedal. Brake 
intervention by the driver will switch off the system immediately. 



4.3 Requirements on Sensors 
4.3.1 Detection Range 

The minimum detection range is defined by the covered distance for an auto- 
matic stop behind a standing vehicle. It is obviously defined within the limits 
of the maximum speed and the maximum comfortable deceleration 
of the driver's own vehicle. The tracking time for a relevant object and the 
response time given by the brake system define the total response time 
The minimum detection range is defined by the following equation: 






1 V 

T, +-^^ + d, 

2 a. 



( 1 ) 



with being a constant value describing a safety distance of 1.5 m. 

The following table shows the minimum detection ranges for velocities of 
30 km/h and 50 km/h assigned to comfortable deceleration values with an 
assumed response time T^=Z s. 



^max 




^max 






2 m/s^ 


3 m/s^ 


4 m/s^ 


5m/s^ 


30 kmh 


35.53 in 


29.74 m 


26.85 in 


25.11 ni 


SO km It 


77.50 m 


51.43 m 


53.39 in 


48.57 in 



Tab. 3. Table of minimum detection range for variable deceleration and dif- 
ferent velocities. 
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A detection range of 30 m with a maximum speed 50km/h and maxi- 
mum deceleration 5 m/s^ implicates a response time of 0.66 s. This con- 

dition is fulfilled easily by the ALASCA® as shown in chapter 2. The following 
diagram shows the required detection range as a function of the response time 
calculated for a maximum velocity of 50 km/h. The ALASCA® system having 
a response time < 0.5 s and a detection range of 30 m could be realized with 
the moderate deceleration value of /*=2.5 m/s^. 



detection range at 50 km/h 



80,00 
_ 70,00 
“ 60,00 
I 50,00 
^ 40,00 
.2 30,00 

I 20,00 
■° 10,00 
0,00 



Fig. 4. Detection range as a function of system response time. 

The driver feels safe if the system notifies and acts on stationary objects quite 
early. At a velocity of 50 km/h and for a response time of 1.5 s the vehicle cov- 
ers a distance of about 21 m before activating the system. At a velocity of 
30 km/h this distance reduces to 12.5 m. For a presumed response time of 
1.5 s a detection range of (1) 50-60 m for a velocity of 50 km/h and (2) 30-40 m 
for a velocity of 30 km/h is necessary. 

In order to cover obstacles appearing during standstill of the vehicle, a detec- 
tion range of 0.5 m is estimated as the minimum range. 




0,50 0,75 1,00 1,25 1,50 1,75 2,00 2,25 2,50 

dead time [s] 



4.3.2 Scan Angle 

The scan angle a (figure 5) is defined to cover the ego lane and both adjacent 
lanes, in order to detect objects cutting-in: 
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— = arctani 

2 I 




(2) 



with Xi being the lane width and being a comfortable stopping distance. 

For a typical lane width of 3.5 m and a comfortable stopping distance of 1.5 m 
the half scan angle a is obtained to be 67°. By choosing this minimum scan 
angle it is ensured that the sensor detects cutting-in objects. 

While driving through curves the sensor will hold contact with the relevant 
objects. For a scan angle of ±75°, curves with a radius >100 m are covered. 
ALASCA® outperforms this range with its theoretical maximum scan angle of 
240°. In reality the scan angle is limited to a still sufficient value of 120° by 
the mounting position on the vehicle. 




ALASCA® delivers an enhanced scan angle in comparison to common ACC 
Systems. Therefore in traffic jams during stand still ALASCA® monitors the 
proximity of the vehicle with high precision for automatic restart. 



4.3.3 Vertical Field of View 

Under S&G conditions enhanced drive dynamics are given compared to ACC. 
Enhanced limits for acceleration and deceleration, commonly -i-3 m/s^ and 
-5 m/s^ are considered. An increased vehicle pitch angle value of -i-0.8° / -2° 
results from this situation. ALASCA® is designed to cover this dynamic part of 
the vehicle pitch angle due to its vertical field of view of 3.2°. 
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The vertical field of view is divided into 4 levels offering a resolution of 0.8°. 
Therefore, it is also possible to distinguish between the road surface and the 
relevant object. During pitching of the vehicle due to road surface irregulari- 
ties ALASCA® will continue to hold contact to the tracked objects. 



4.3.4 Angular Horizontal Resolution 

The new advanced task of the ACC S&G is to offer: (1) braking in front of sta- 
tionary objects or (2) driving past parked vehicles at the lane side - demanding 
new requirements for the horizontal angular resolution. The requirements for 
the resolution are defined by the request of detecting the lane edge, the later- 
al position of the relevant object and the object width. 

An example is shown in figure 6 for a typical situation with the ego vehicle in 
the centre lane and a parked car in the neighbouring lane. For passing the 
parked car on the right side a safety distance of d^'^30 cm is demanded. The 
resolution of the y-coordinate should be at least half of the safety distance. For 
safe passing this requirement translates into an angular horizontal resolution 
of 0.3° in conjunction with a detection range of 30 m. ALASCA® fulfils the 
request with its horizontal angle resolution of 0.25° to 1°. 




Fig. 6. Illustration of the angular horizontal resolution. 



4.3.5 Object Data 

The following data of all relevant objects are delivered by the sensor system to 
enable sufficient brake and motor management. 



Distance - The distance measurement resolution is 1 cm with a sigma of 5 cm. 
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Velocity - Velocity data are necessary for ACC-control and the detection of 
stopping and stationary objects, respectively. The resolution is 0.5 km/h mini- 
mum. 

Acceleration - In the lower velocity range the acceleration of the relevant 
object is necessary for ACC control. Lateral acceleration values are necessary 
to detect cut-in and cutout manoeuvres. 

Position - To select the relevant object for brake and motor management con- 
trol, the object position in the (x, y)-coordinates of the right and left boundary, 
the (x, y)-coordinates of the object centre, and the object width are necessary. 



4.4 System Concept 




Fig. 7. Basic elements of an autonomous S&G system containing the sen- 
sor fusion of IDIS® and ALASCA® and engine and brake manage- 
ment controlled by the Central Processing Unit (CPU). 

The following features are fundamental for an autonomous S&G system and 
controlled by the Central Processing Unit (CPU) shown in figure 7: 

► Scanning of the vehicle environment in front and in the neighboring 
lanes. 

► ALASCA® calculates the object outline from the scan data and appro- 
priate algorithms. The contour lines serve to extract and classify the 
objects. In addition, the distance, direction and relative velocity data of 
each object are supplied for further signal processing. 

► Adopting of the relevant object from ACC-Software from IDIS® or track- 
ing of relevant obstacles in the vehicle driving direction by ALASCA®. 
The object data are transmitted to the central processing unit (CPU). 
The CPU selects the relevant objects. 

► Automatic adaptation of the speed setting to the distance and speed of 
the vehicle driving in front if necessary, up to a complete standstill. 
Based on the current data of the vehicle motion, the CPU directly con- 
trols the engine and the brakes. 
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► Auto restart under following conditions: 

► All passengers are on board and vehicle doors are closed. 

► No object is located between the driver's own starting vehicle and the 
tracked vehicle. 

For safety reasons the driver is able to override the automatic control at ACC 
S&G. 



5 Additional Applications 

5.1 Pedestrian Detection and Protection 



In the future the number of fatal accidents of pedestrians will be reduced by 
active and passive systems on the vehicle. Actively reacting systems require a 
precisely measuring sensor like the ALASCA® in order to provide a reliable 
database for introducing contour changes to the vehicle (lower bumper, active 
hood). Therefore sensorial systems are needed that detect and classify pedes- 
trians crossing the path of the ego vehicle. ALASCA® delivers high-resolution 
range images in a wide horizontal angular range. Combined with advanced 
sensor signal processing algorithms for high speed object detection, tracking 
and classification this sensor provides the data required for future active 
pedestrian protection systems. 




Fig. 8. A video image (left image) and computed classification data (right 
image) of a scene showing a pedestrian just steeping onto the road 
from behind a parking car. 
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Pedestrians can be detected, tracked and classified even at larger distances as 
shown in figure 8, which allows their observation for a longer period of time 
in the scene. That allows a robust detection of potentially harmful situations. 



5.2 Precrash 

This safety system requires the detection and evaluation of an imminent colli- 
sion. This system provides a larger time slot -compared to impact operated 
sensors- to activate safety measures like seat belt pre-tensioners, intelligent 
airbags, or the pretensioning of the brake booster. Reversible systems can be 
activated at a stage when a collision can stUl be avoided. 

Upon the identification of an obstacle within the driving path the Time-to- 
Collision, Type of Collision Partner and Point of First Contact are calculated. 
Figure 9 shows a test scenario where the ego-vehicle is approaching an obsta- 
cle (box) of 40 cm width. In the video image, the scan data of the four ALAS- 
CA® scan planes are displayed in an overlay mode. 




Fig. 9. Video image (left) and schematic (right) of a Pre-Crash scenario. 
Note the 4 scan planes of the laserscanner on the target-box. 



Figure 10 shows crash-probability and time-to-collision, both as function of 
scans (on same scale). It can be seen from the graphs that while the number 
of scans increases the collision probability rises from 50% at scan #1 to 100% 
at scan ^10 with a time-to-collision of approx. 100 ms remaining at the point 
of inevitable collision. 
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Fig. 10. Collision probability (top) and time-to-collision (bottom) of the Pre- 
Crash scenario from figure 8. 



5.3 Parking Assistant 

While driving on a road with potential parking slots on the side these will be 
evaluated and mapped by the parking assistant. ALASCA® measures with a 
fine contour resolution the parking slot as well as different types of curbs. The 
complexity of the parking assistant depends on the realization. A first step of 
realizations might be a simple hint for a parking slot, like yes/no statements. 
The next step might be automatic steering into the parking slot with brake 
control of the driver. In future automatic parking will be realized. 



5.4 Turning Assist for Trucks 

Cyclists and people located in the blind spots of trucks are often overlooked by 
the driver because of the lack of monitoring systems. This results in a large 
number of fatal accidents. The implementation of a turning assistant will be an 
important step to reduce the number of these accidents. It is realized by inte- 
grating an ALASCA® into the front right corner of the truck. The system mon- 
itors the area on right side of the truck with a scan angle of >90°. Thereby 
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vulnerable road users are detected. In case of a dangerous situation a warning 
signal (e.g. a red no-go sign in the side mirror) will be given to the driver. An 
advanced version with a scan angle up to 240° would be able to extend the 
field of view to the area in front of the truck. 



6 Experimental Vehicles 
6.1 Integration Analysis 

For several vehicles from different manufacturers, a design integration analy- 
sis was carried out. Slight modifications of the front end of the car allow invis- 
ible mounting of the laser scanner. Integration can be realized in all types of 
cars. As an example for the front centre integration a car equipped with an 
integrated ALASCA® is shown in figure 11. The LIDAR system IDIS® is also 
shown in its mounting position. The sensor fusion of both systems is of par- 
ticular interest for the application S&G. 




Fig. 11. Integration of ALASCA* into the vehicle front end. 



ALASCA® is mounted under the bumper in a centre position. For design rea- 
sons and protection ALASCA® is enclosed in a housing covered by an IR-trans- 
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mitting window. This integration position permits a horizontal scan angle of up 
to 120°. The application S&G and AEB have been demonstrated with a test 
vehicle equipped with this integration solution. 

6.2 Electronic Configuration 



Fig. 12. Vehicle architecture of sensors and data bus system. (1) ALASCA®, 



The output signals of ALASCA® (1) are 

► AvEiilabilitY judged upon self-test. 

► AEB activation. 

► Object. 

► Object classification. 

► Object velocity. 

► Object position (x, y). 

► Object acceleration. 

These data are transmitted via CAN to the central processing unit CPU (4). The 
CPU also receives the object data generated by the ACC System (e.g. IDIS® (2)), 
the vehicle velocity and acceleration, etc. Depending on the application and 
the motor data, the CPU controls the engine (3) and the brakes (5). 




(2) IDIS®, (3) engine, (4) Central processing unit, (5) brakes. 



The experimental vehicles are equipped with both applications, AEB and S&C. 
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7 Conclusion 

In this paper an ADAS sensor technology in combination with its applications 
was presented. It was described in detail, how the laserscanner ALASCA® sup- 
ports Automatic Emergency Brake, Stop and Go, Precrash, Parking Assistance, 
Pedestrian Detection and lUrning Assist. In 2004 vehicles equipped with 
ALASCA® will be available for OEM-testing. 
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Abstract 

In established driver assistance systems each application relies on its 
own sensor, which observes the vehicles environment. Advanced 
driver assistance systems (ADAS) have increasing demand for sever- 
al sensor systems. The described laserscanner and video based sen- 
sor fusion approach serves as a general platform for multiple active 
safety and comfort applications. By fusing these sensors, a large field 
of view is obtained and the certainty and precision of the estimates 
in the relevant regions is increased significantly. Robust vehicle posi- 
tion measurements and classification, lane estimation as well as 
pedestrian recognition enable a broad support for applications such 
as lane departure warning, automatic emergency breaking, auto- 
matic cruise control for traffic jam situations (ACC Stop&Go), 
PreCrash, pedestrian protection and low speed following. 



1 Introduction 

Recent projects on driver assistance systems have focused on applications such 
as Pre-Crash (Chameleon) [1], ACC Stop&Go (Carsense) [2] and recognition of 
vulnerable road users (PROTEGTOR). These advanced driver assistant systems 
(ADAS) have increasing demand for several sensor systems, which are not only 
complementary but also redundant. Some research has therefore been focused 
multi-sensor fusion [2-5]. The aim of such a step is to provide a fused descrip- 
tion of the traffic scene surrounding the vehicle, which is relevant for ADAS, 
but not specific to a certain application. This fusion system incorporates the 
data of the diverse sensors into a single description. Thus the field of view of 
a single sensor is enlarged, the certainty and precision of the estimates is 
increased and additionally this system design is economically efficient, as dif- 
ferent applications share a set of sensors. 

In section 2 the motivation leading to the use of sensor fusion systems is 
explained and our approach specified. Section 3 deals with the laserscanners 
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object tracking and classification as well as the pedestrian recognition. The 
image processing is explained in section 4. In section 5 some results of our sen- 
sor fusion system are presented and section 6 concludes the paper. 



2 Sensor Fusion 

2.1 Introduction 

Established advanced driver assistant systems, such as Lane Departure 
Warning (LDW) and Automatic Cruise Control (ACC) have a common architec- 
ture, as shown in Figure la. The application is implemented for a special sen- 
sor configuration and therefore depends on these sensors. With an increasing 
number of applications which share a set of sensors, the sensor dependent part 
of the software would exist several times, i.e. found in each application (see fig- 
ure lb). Thus emerges a strong motivation to separate the sensor specific part 
of the system and use it as an interface between the sensors and the applica- 
tions (see figure Ic). The sensor fusion module is responsible for collecting 
measurements, interpreting them sensor specifically and incorporating them 
into a unified, consistent description which is then forwarded to the applica- 
tions. The sensor fusion must be in part sensor specific in order to obtciin a 
maximal profit of each sensors measurement. 




a 



b 



c 



Fig. 1. Driver assistant system architectures, (a) A single application 
which depends on one or several sensors, (b) Several applications 
each based directly on one or several sensors, (c) Possible future 
advanced driver assistant system architecture with common sen- 
sor fusion layer, thus separating the application from sensor spe- 
cific implementations. 
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2.1 General High-Level Sensor Fusion 

A general architecture of a centralised high-level sensor fusion system con- 
tains the basic components shown in figure 2 [6]. The environment description 
is composed of a set of objects, each of which is defined by an object state. 
There are different object models for trucks/busses, passenger cars, (motor) 
bikes, pedestrians and for stationary objects. Additionally the road is modelled 
as well as the ego-vehicle. 

The objects of the environment description are predicted in order to synchro- 
nise them with the incoming sensor data. The object states are transformed 
into the parameter space of the sensor data by the use of inverse sensor mod- 
els and are associated with the measurements. In case of an established asso- 
ciation the sensor data is integrated into the object state. An object manage- 
ment handles the generation and deletion of objects in the environment 
model. 



Applications 

A 




Sensor data 



Fig. 2. General architecture of a centralised high-level sensor fusion sys- 
tem which works as an interface between sensors and applications. 

As mentioned, the sensor fusion depends on the sensor's characteristics. The 
sensor specific modules are the inverse sensor model and the association. 
These are the only components of the sensor fusion layer which need to have 
some knowledge of the sensor's characteristics. The integration and prediction 
may be performed using a Kalman-filter. 
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2.2 Synchronization 

Sensor fusion combines several sensor measurements in order to obtain an 
enhanced object state estimation. The association and integration of sensor 
data requires its synchronisation with the environment description. The envi- 
ronment description is predicted for the time of measurement. Thereafter the 
sensor data can be associated and the object states updated. In order to obtain 
a precise synchronisation a sufficiently accurate global time for all sensors and 
the fusion system is necessary. 

In order to obtciin a time-consistent object state estimation the measurements 
have to be integrated in the order, they were acquired. This is a crucial con- 
straint for a sensor fusion system. Depending on the synchronisation strategy, 
measurements have to be buffered for chronological sorting thus introducing 
a latency into the fusion system. For very time-critical ADAS such as PreCrash 
or automatic emergency brake, the overall system latency is of high interest. 
The advantage of such ADAS is drastically reduced if their reaction time is too 
long with respect to the high speed of passenger cars and therefore short time 
to contact in extreme situations. The aim is to minimise the systems latency 
which in our case is the latency introduced by the sensor fusion process. 
Different synchronisation strategies lead to varying worst-case sensor fusion 
latencies. As shown in [7] using synchronised sensors the latency introduced 
by the chronological sorting can be minimised. An additional advantage of 
using synchronised sensors is that the latency is constant over time and there- 
fore determined. 

In our approach we use two environment sensors, a laserscanner and a video 
camera. In order to synchronise the sensors, the camera is triggered at those 
points in time, when the rotating laserscanner head is aligned with the direc- 
tion of the optical axis of the camera. 



2.3 Calibration of Laserscanner and Camera 

The sensors are calibrated in order to enable not only a temporal alignment 
given by the synchronisation but also a spatial alignment. Figure 3 shows an 
image where the laserscanner data is projected into the image domain. By 
means of an accurate synchronisation and calibration, image regions can be 
associated directly with laserscanner measurements. Therefore it is possible to 
assign certain image parts a distance, which is a major advantage of this fusion 
approach. In figure 3 the laserscanner measurements on the back of the pre- 
ceding car fit very well with the image taken by the camera. The precision of 
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the association between the scans and the image can also be seen on the bol- 
lards delimiting the area of the road works. 




Fig. 3. Laserscanner measurement in the bird view. The different layers of 
the multilayer laserscanner are colour coded. In grey the field of 
view of the camera (left). Laserscanner data projected into the 
image domain (right). 



2.4 System Architecture 

The sensors used for our sensor fusion approach are a laserscanner with four 
scan planes, a monochrome monocular camera and an Inertial Navigation 
System (INS). The laserscanner is mounted at the front bumper of our test 
vehicle and the camera behind the front screen beside the inner mirror. The 
field of view of the laserscanner compared to the field of view of the camera 
is shown in figure 3. The horizontal view of the laserscanner is with 
180° much larger than that of the camera with 28°. However, the horizontal 
resolution of the laserscanner is with 0.25° (0.5°) at 10 Hz (20 Hz) much lower 
than the resolution of the 640 x 480 pixels image with 0.045°. 

Both sensors are combined in the fusion system in a way in order to enable a 
high amount of synergetic effects. The laserscanner tracks and classifies vehi- 
cles, motorbikes and pedestrians in the 180 degree environment in front of the 
ego vehicle. The image processing unit estimates the lane and the position of 
the ego vehicle in the lane. Another image processing module refines the lat- 
eral position and velocity estimation of the objects tracked by the laserscan- 
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ner. At times when the objects exceeds the object detection distance of the 
laserscanner the object can be tracked for a while in the image. 

The high-level fusion combines the ego motion estimation, the lane recogni- 
tion and the detected objects into a unified description. Based on the high-level 
fusion, applications could detect lane departures of the ego vehicle, obstacles 
in the driving path, determine objects of interest for ACC and Low Speed 
Following, as well as their lane changes or detected potentially endangered 
pedestrians. 




Fig. 4. System architecture involving different levels of sensor fusion. 



3 Laserscanner 

There is a need for sensorial systems which are able to observe the vehicles 
environment in order to detect and classify endangered pedestrians and to 
track vehicles, laserscanners deliver high resolution range images in a wide 
horizontal angular range. Combined with advanced sensor signal processing 
algorithms for high speed object detection, tracking and classification, these 
sensors are able to provide the sensorial data required for future active safety 
systems. 









Sensor Fusion for Multiple Automotive Active Safety and Comfort Applications I 143 



The multilayer laserscanner (LD ML) used for environment observation uses 4 
scan planes instead of one which allows for a compensation of the pitch angle 
of the ego-vehicle [8], 




Fig. 5. Multilayer laserscanner LD ML. 



3.1 Object Detection 

The distance measurements generated by one revolution of the rotating head 
(scan) are divided into segments, which are assumed to belong to the same 
object. The contour of a segment representing a vehicle consists typically of 
one or two straight lines. The appearance depends on the orientation and dis- 
tance of the object in the laserscanner's co-ordinate system, as shown in fig- 
ure 6. Pedestrians typically appear as a small cluster of measurement points. 



3.2 Object Tracking 

In order to track objects close to the ego vehicle the predicted object states of 
these objects have to be associated with current measurements (segments). 

In general the estimation of the ego-motion under defined driving conditions 
is based on the vehicle's INS-Data [8]. A reliable ego-motion estimation under 
both, defined and undefined driving conditions, can be determined by using 
the relative velocity of static objects in the vehicle's environment such as 
posts, reflection posts and trees [9]. The ego-motion of the test vehicle is com- 
pensated before segments are assigned to object tracks in order to obtain a 
more reliable and robust tracking. The typical Kalman filter cycle is performed 
in three steps. 
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First, the state vector x and the state error covariance matrix ~P(k) are pre- 
dicted and the ego motion compensation is performed. The object model used 
is a single-point-mass model, which simply represents an object as a mass that 
can move in the 2D space without any cinematic constraints. For each model 
class, a constant velocity is assumed. Therefore accelerations of the objects are 
considered in the model covariance. Different acceleration limits depending on 
the object's class lead to the definition of class specific covariances. Therefore 
accurate gating areas are obtained, as shown in figure 7. For more details see 
[ 10 , 11 ]. 
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Fig. 6. Segments with associated video image sorted by distance. The seg- 
ment contour and the number of scan layers (coded in different 
colours) depends on the distance. 

Second, segments are assigned to object tracks, which will be discussed in the 
next section. 

In the third step a new measurement (object reference point) is incorporated 
into the a priori estimate of the Kalman filter, thus improving the a posteriori 
estimate. 



3.4 Object Association 

The data association is responsible for the assignment of segments to object 
tracks. The association process can be divided into three sub-processing 
stages. The computation of the gate, checking of the gate criteria and the 
assignment of segments to the objects. 
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First, gating is used as a screening mechanism in order to determine which 
segments are valid candidates to be associated with an object track. Gating is 
performed primarily to reduce unnecessary computation by the association 
algorithm. A rectangular gate is used whose orientation is determined by the 
absolute velocity vector of the object. The size of the rectangular gate is adapt- 
ed to the vehicles dimensions (width and length /y) and to the quality of the 
state estimation (figure 7). 




Fig. 7. Gate of the association step to assign the matching segment. 

If several segments meet the gate criterion, an object reference point is calcu- 
lated for each segment in order to determine the closest segment or segment 
combination. Under real world conditions, it can be difficult to accurately iden- 
tify an object reference point, because of the changing contour and conditions 
of the segments, as shown in figure 9. 

In order to determine the object reference point three different methods are 
implemented and evaluated, which will be discussed in the following. They are 
compared and evaluated based on simulated data, which enables an objective 
comparison. 
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Fig. 8. Object reference point calculated by the centre of gravity approach 
(left) and the two line approach (right). 



Centre of Gravity Approach 

This approach calculates the object reference point as the mean of all segment 
points, as shown in figure 8 (left): 



« "pjlW 

point t.l 



( 1 ) 



Where is the number of points of a segment and pi(x,y) is the position of 
a point in the laserscanner's co-ordinate system. This strategy considers all 
points of a segment for the computation of the object reference point and is 
used for all objects, except for vehicles. Considering a passing vehicle the 
object reference point moves inside the object contours, as shown in figure 9, 
which would lead to inaccurate velocity estimates. 




Laserscanner 



Fig. 9. A car passing in front of the ego-vehicle, which is equipped with a 
laserscanner. The blue dots represent the object reference point, 
which is calculated by the centre of gravity approach. 
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Object-Box Approach 

This approach is designed to track vehicles. Vehicles are represented by a rec- 
tangular box with adaptive length and width, the so called adaptive object 
box, which is calculated from the segment's parameters. 

The points of a segment are sorted clockwise by their angle, where the first 
point f of a segment is the point with the smallest angle and the last point 1 is 
the segment-point with the largest angle. The measurement point of the seg- 
ment with the shortest distance to the scanner c is called the closest point. A 
straight-line-object is represented by a segment with strmght line contour (see 
figure 6, distance 18 m, 42 m and 79 m and figure 10). In consequence, an 
angle-object is represented by a segment with angular shape (see figure 6, dis- 
tance 10 m). 

TWo strategies were developed to compute the rectangular box. It is differen- 
tiated between objects of known velocity and objects without known velocity. 
Moreover, it is distinguished between straight-line-objects and angle-objects, 
especially for vehicles like cars and trucks. 

Straight-line-objects 

The heading of a moving object is determined by the velocity vector. 
Comparing the heading with the vector from the first to the last point 
a segment, the length and width of the tracked object can be calculated. Using 
a priori knowledge the adaptive object box with the actual length or width is 
computed. 

By means of the normal vector 
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the object reference point for vehicles moving radial to the laserscanner is 
computed by 
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If the vehicles are moving tangential to the laserscanner, the object reference 
point is given by 
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where represent the length and width of the object, respec- 

tively. 




Fig. 10. Different representation of a vehicle by a segment, depending on 
the angle of view to the object. 



Straight-line-objects without any velocity are always represented by a seg- 
ment which corresponds to the length of the tracked object. The object box 
will be aligned on the segment with the actual length. If a priori knowledge is 
available the object box has the maximum width of the tracked object that 
ever occurred. The object reference point is the centre of gravity. 
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Angle-objects 

For moving objects the heading of the object can be calculated from the veloc- 
ity vector. Comparing the heading with the vector from the first to the closest 
point u p(^ and the vector from the last to the closest point MpQ it can be deter- 
mined which line of the segment represents the width and which line the 
length of the tracked object. Using the vector from the first to the last point of 
a segment and the velocity vector v as shown in Figure 11, the angle 
between both vectors can be computed and thus the length and width of the 
object. 




Fig. 11. Computation of length and width of the vehicle using a) the veloc- 
ity aligned vector (left) b) without known velocity, the vectors 
between the first, last and closest points (right). 

Knowing the length and width of the object, the computation of the object ref- 
erence point can be easily done. As shown in the following equations it has to 
be differentiated which line of the angle-object is parallel to the velocity vec- 
tor v. 
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In the case of a non moving object, it depends on the object class which 
method is used to compute the object reference point. Objects that represent 
vehicles can be described by an object box even without velocity. However, it 
is not possible to determine a reliable length and width of the object, therefore 
the longer line is supposed to be the length and hence the shorter line to be 
the width. The object reference point can be computed by means of the vec- 
tors u and Mpp; which are supposed to be almost orthogonal to each other. 
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Two Line Approach 

The two line approach uses one Kalman filter (L) to track the length and one 
Kalman filter (W) to track the width of the object. As shown in figure 12, the 
measurement points of a segment are divided into two straight lines. The divi- 
sion of the segment is done analogous to the width and length computation in 
the previous section. 

Using the two line approach, also two object reference points are required to 
update the Kalman filter (L, W). In this case the object reference points are the 
geometrical centres of the lines. The motivation for this approach is that 
depending on the orientation in the laserscanner co-ordinate system, the seg- 
ments consist of either one or two straight lines, as shown in figure 12. If there 
is only one straight line, just the appropriate Kalman filter will be updated. 
Whenever both straight lines, a so called angle-object is detected, both Kalman 
filter are updated to track the object. 

To compute the velocity of an object, both Kalman filter elements for width 
and length are weighted depending on their performance. The weighting is 
based on the innovation vector &(k). 



where Ti(k) is the temporary generated measurement vector and n~(k\k-l) is 
the predicted state vector of the previous step. This leads to a weighting fac- 
tor g(k) 



Consequently, the computation of the velocity vector v is done by weighting 
both the velocity vector of the width, v'*^, and the length, v^, with the factor 
g(k) as follows 



a(k) = y(k) - C ■ x‘ (k | k - 1 ) 
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g'^(k) + g‘-(k) 



v(k) = g(k)-v'*'(k)-K(l-g(k))-v^(k) 



( 9 ) 
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Fig. 12. The measurements at a vehicle are divided in two straight lines 
(left). A vehicle can be represented by one or two straight lines, 
depending on the angle of view (right). 

To handle the changing of the segment contour, the change of the size (length 
or width) of an object is considered. As illustrated in figure 13, the real move- 
ment of the object displayed with a blue arrow and the movement of the object 
reference point, represented by black arrows, differ depending on the angle of 
view. 
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Fig. 13. Change of the size (here: length) of an object due to different 
angles of view which causes a change of the object reference point. 

In this case an imaginary movement of the object reference point would occur, 
which is a result of a change of size of the current object. To avoid the imagi- 
nary movement of the object, an object reference point is calculated, which is 
at the same point in the object co-ordinate system as in the previous scan. 




152 I Safety 



The computation of the object reference point of the innovation o'^^yconsists of 
the actual object-length the maximum occurred object-length 

and the vector u pQ 
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After the innovation the real object reference point set into the state 

vector of the Kalman filter to obtain a more reliable prediction in the next 
Kalman fUter cycle. 
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The introduced methods are compared and evaluated based on a scenario with 
simulated data, where a car passes the laserscanner, as displayed in figure 9. 
The velocity of the car, calculated by the tracking algorithms, is a good indi- 
cator for the quality of the different tracking techniques. Therefore the 
obtciined velocities of the different approaches are compared with the prede- 
termined velocity of the simulated object, as shown in table 1. 



Method 


Standard deviation 3 o I 


Centre of Gravity approach 


7.7 km/h 


Object box approach 


1.1 km/h 


Two Line approach 


0.3 km/h 



Table 1. Evaluation of the different methods with simulated laserscanner 
data for a passing car. 

However the evaluation on simulated data is only the first step in the devel- 
opment of new algorithms. Ongoing evaluations on real data show that real 
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vehicles are tracked very precise as well. As in real scenarios the true veloci- 
ty is not known, estimations are done to compare these with the calculated 
velocity of the object. Without doubt the calculated velocity of vehicles based 
on real data would not be as good as the simulated results, but the estimated 
accuracy of about +1 km/h in first investigations with the two line approach is 
convincing anyway. 



Segment to Object Assignment 

There are several well known procedures to assign segments to objects. The 
usual assignment method is the global nearest neighbour method. However, 
there are more sophisticated methods for the assignment problem. 

In the current algorithm an assignment matrix, as shown in figure 14, is used 
in conjunction with the Hungarian method for the assignment problem. The 
columns of the assignment matrix represent the objects and the rows repre- 
sent the segments. Every time a segment is within an object-gate, there is an 
entry in the assignment matrix at the equivalent position. 



objects 





not unique segment- 
□ ■^r~ object assignments 

□ □ 




^ • 

unique segment- 
object assignment 



number of elements in a column 



Fig. 14. Assignment matrix to combine the matching segments to the 
object tracks. 

Unique segment-object assignments can be obtained in the assignment matrix 
and an instantaneous assignment can be carried out. This is the case, if only 
one segment is within a gate. These unique assignments are cancelled in the 
assignment matrix and a sub-matrix with only non unique assignments is gen- 
erated. 
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The sub-matrix with non unique assignments has to be further processed. For 
that we use the Hungarian method, which returns the best possible assign- 
ment. The Hungarian method solves all non unique segment-object assign- 
ments. 



3.5 Object Classification 

The object classification in general is based on typical a priori known object- 
outlines (static data) of typical road users, such as cars, trucks/busses, motor- 
cycles/bicycles and pedestrians. Additionally, the history of objects classified 
earlier and the dynamic state variables of the tracked objects are taken into 
account to improve the classification performance and its reliability. 

In case there is not enough information for an object classification a hypothe- 
sis is generated, based on the objects current appearance. The temporary 
assignment is valid, as long as there is no violation of a limiting parameter. 
However the classification is checked every scan to verify the assignment of 
the specified class. 



3.6 Pedestrian Recognition 

The strategy to classify pedestrians is based on the following idea: An object 
that looks like a pedestrian with respect to its outline is classified as a pedes- 
trian only, if the object is presently moving or has moved during the period of 
tracking in the past. Otherwise it will be classified as a non moving obstacle. 
However, if the non moving pedestrian, classified and tracked as a non mov- 
ing obstacle, is located in the predicted path of the ego-vehicle, the situation is 
rated to be dangerous anyway. 

If an object is detected for the first time, the classification result is based sole- 
ly on its outline. As in this case a small object can be either a pedestrian or a 
small static object, such as a pole or a lamppost, the classification result is 
'ped?'. As long as a small object is not moving towards the road, there is no 
hazard implied, neither for the vehicle nor the object itself. If a small object is 
moving, it is immediately classified as a pedestrian, based on the previous ini- 
tial classification 'ped?' and its absolute velocity [12]. 

Slow pedestrians, which are able to change their moving direction in a split 
second, are tracked with the simple single-point-mass model, with the same 
iimiting parameters to all directions of movement. If the pedestrian is moving 
faster (> 20 km/h), it can neither change the moving direction nor increase the 
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speed rapidly, just deceleration could be faster. For that reason different lon- 
gitudinal and lateral limiting parameters are used to model this behaviour. A 
running pedestrian has a similar model as a bike with the same speed. 




Fig. 15. Overview on the different test scenarios. 

The pedestrian recognition algorithm based on range images of a laserscanner 
was investigated in different test scenarios. It is important that the system 
detects and classifies every pedestrian in the region of interest (ROl) at a high 
confidence level. The performance is assessed by the 'true positive' rate. The 
obstacles, which are classified wrongly as a pedestrian are counted resulting 
in a 'false positive' rate. 

In scenario 1 the ability to detect and classify pedestrians at different distances 
and angles was tested (figure 15). Here, a pedestrian walked in an unstruc- 
tured environment (free parking space). The ROl was defined with a longitu- 
dinal range from 0 m to 60 m and a lateral range of ±8 m in front of the ego 
vehicle. The tests performed in scenario 1, which comprises more than 3000 
scans in total, results in a true positive rate of 100%. The false positive rate 
was 0.0. This is not really surprising as no other objects existed within this 
test. 

Scenario 11 was performed in an urban environment. The ego vehicle was driv- 
ing with different velocities of up to 50 km/h. A pedestrian is walking along 
the road. The ROl is defined in a longitudinal range from 0 m to 60 m and a 
lateral range of ±5 m in front of the ego vehicle. The pedestrian walking along 
the road were detected and classified all the time in the ROl (true positive rate 
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of 100%). Sometimes other objects were classified as a pedestrian at the 
boundary beside the street (false positive rate of 20%). By introducing a fUter 
that suppresses the output until the object was classified 3 times within the 
same class a false positive rate of 0% can be achieved even in this scenario. 
However, due to this filter process, potential pedestrians are output to the 
application with a delay. The total delay depends on the scan frequency of the 
laserscanner. At 10 Hz scan frequency this delay would be about 300 ms, 
using 40 Hz scan frequency the delay would be about 75 ms. However the 
update rate after initial classification is 100 ms, 25 ms respectively. 

The last scenario III was carried out on a test field, where tubes mark the 
boundary of the road, as shown in figure 15. A pedestrian is stepping onto the 
marked road initially occluded by a parking car. The classification of the pedes- 
trian crossing the street was always done correctly in this test case (true pos- 
itive rate of 100%). No other objects than pedestrians were detected belonging 
to the class pedestrian (false positive rate of 0%). However, two scans are need- 
ed to compute the velocity and perform the first classification result after the 
pedestrian enters the field of view of the laserscanner from behind the park- 
ing car. 





True Positive 


False Positive 


False Positive 
(with additional filter) 


Scenario 1 


1,0 


0.00 


0,00 


Scenario II 


1.0 


0.20 


0.00 


Scenario III 


1.0 


0.00 


0.00 



Table 2. Summary of the classification results. 

The classification results of the first approach to recognise pedestrians as dis- 
cussed above are summarised in table 2. They illustrate a good performance 
for less structured environment. However, there are still scenarios which will 
be a challenge for pedestrian recognition, e.g. highly frequented urban sce- 
narios with lots of pedestrians walking close together. Also the differentiation 
of a post or a tree and a non moving pedestrian in a range image of a laser- 
scanner can not be done up to now. 
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4 Video 

One advantage of the laserscanner measurement principle is, that only there, 
where the laser heam is reflected by an object, a measurement is performed. 
Therefore the laserscanner is very reliable in object detection. Additionally the 
distance is measured very accurately. Both characteristics are weak points in 
monocular image processing approaches. On the other hand a video image 
exhibits a higher angular resolution than a laserscan. Therefore we take the 
advantage of both systems and design an image processing algorithm which 
is attention driven by the laserscanner's object detection and which refines 
the lateral position and velocity as well as the width measurements. 





Fig. 16. Bird view of a typical scenario on a highway. In black the laser- 
scanner measurements. Three cars are detected by the laserscan- 
ner. In red the field of view of the camera. 
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4.1 Region of Interest 

In figure 16a typical highway scenario is shown. Three cars are detected by 
the laserscanner. TWo cars are in the overlapping field of view of both sensors 
and one car is only seen by the laserscanner. The objects detected by the laser- 
scanner are projected into the image domain via a homogenous transformation 
given by the calibration. No temporal alignment is necessary, as the image 
acquisition is synchronised with the laserscanners measurement cycle. 




Fig. 17. In violet the ROIs given by the laserscanner. The U-shapes show 
the width and position of the cars refined by the image processing. 



A region of interest (ROI) in the image is calculated from the objects width, 
measured by the laserscanner and the type of the object, classified by the 
laserscanners object recognition (see figure 17). 



Object Recognition 

The ROI of each object is searched for features which exhibit the presence of 
a vehicle. Depending on the classification of the laserscanners object recogni- 
tion into cars and trucks, different features are searched for. Using adaptable 
thresholds the applied image processing is widely independent of illumination 
changes. 
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Shadow 

A first feature is the shaded underhody of vehicles. In day time scenarios this 
shadow is almost independent of the illumination. For cars in the far field 
(>50 m) this dark area takes a special shape. As shown in figure 18a the 
wheels and a few image rows above the wheels exhibit dark grey values. 



01 ( 0 ( 

Fig. 18. ROI of image (a, left). In green the wheel pattern hypothesis in the 
binary image (b, from left). Vertical, horizontal and combined 
edges in the image (c, three pictures at right hand side). 

In order to search for this pattern of dark area, a binary image is calculated 
from the ROI. In figure 18b all pixels which are below a threshold are shown 
in orange, the others are drawn in black. Possible shapes and sizes of the tem- 
plate's appearances in the image domain are motivated by the possible width 
of passenger cars as well as the real sizes of their wheels. The templates are 
scaled with respect to the distance of the vehicle measured by the laserscan- 
ner. Therefore the wheel pattern is searched for in every row of the ROI tak- 
ing minimal and maximal possible parameters of width of cars and their 
wheels into account. Every match is taken as a hypothesis. In figure 18b the 
green horizontal lines show the regions a matching wheel template was found. 
Then the ROI is searched for shadow regions which are above the hypothesis 
of the wheel pattern and stem from the dark areas beneath cars. For all 
hypothesis combinations a probability is calculated. If the probability of the 
best hypothesis exceeds a threshold it is taken as a match for the recognition 
of the wheels. From the position and width of the wheel pattern together with 
the distance measured by the laserscanner, a very accurate position and width 
of the object in the 3d ego-vehicles co-ordinate system can be calculated. As 
this wheel pattern is only exhibited by cars in the far field other featured are 
taken into account. 
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Image Edges 

Figure 18c shows the horizontal, vertical and combined edge estimates of the 
ROl. The ROI is searched for strong support of vertical edges for the estima- 
tion of the left and right edges of the vehicle. Different hypothesis are weight- 
ed with the expected width and position of the vehicle in the image domain 
calculated from the laserscanner measurements. 



Symmetry 

The last feature taken into account is the symmetry of the image extent in the 
ROl. The symmetry operator is used in order to check different possible 
hypothesis of left and right edge pairs for their probability of representing the 
true vehicle edges. 



Feature Tracking 

In order to stabilise the estimates a feature tracking approach is applied to the 
vehicle in the ROl. Additionally this tracking approach delivers an estimate for 
the lateral velocity if scaled with the distance measured by the laserscanner. 



5 Results 

A major problem of the first generation of radar based ACC systems is the 
inaccuracy of the estimation of the lateral offset of preceding vehicles. 
Therefore lane changes of objects in front of the ego-vehicle are detected only 
when the vehicle is already completely on the target lane. Therefore in case of 
an object cutting into the ego-vehicles lane, the ACC system often reacts so ate, 
that the user is forced to take over. 

The proposed fusion was tested concerning the accuracy of the estimation of 
the lateral offset. Figure 19 shows the distance x of a preceding car over time. 
The car was detected at every instance in a distance of 50 m and slightly above 
100 m. The second plot shows the width of the car estimated by the laser- 
scanner compared to the fused estimate and the third plot shows the estimat- 
ed lateral offset y. No tracking was applied in this case, every parameter was 
derived from a single scan and image. 

The estimates of width and lateral offset of the laserscanner exhibit a certain 
discretisation which originates from the angular resolution. The image pro- 
cessing algorithms are able to refine both estimates of width and lateral offset. 
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Fig. 19. Distance x, the width and the lateral offset j/ of a preceding car. 



6 Conclusion 

A sensor fusion system was presented which combines the object tracking of 
a laserscanner and of a video system as well as lane recognition, performed by 
an image processing unit. The environment recognition in the near field 
(<50 m) is mmnly based on the very accurate laserscanners object tracking 
which covers an angular field of view of 180°. In the frontal far field the lat- 
eral offset estimate is refined by an image processing module. The fused esti- 
mate exhibits a high precision which is necessary for early lane change detec- 
tion of preceding vehicles. The environment description is application inde- 
pendent and can be used by multiple automotive active safety and comfort 
applications. 
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Dealing with Uncertainty in Automotive Collision Avoidance 



J. Jansson, Volvo Car Corporation 



Abstract 

Modern automobiles incorporate more and more active driver assist 
system that uses sensors and microprocessors to control the vehicles 
dynamics. This paper discus a driver assist system that performs 
autonomous braking when the vehicle is close to colliding. Decision 
making in such systems is inherently uncertain, due to the sensors' 
measurement uncertainty and the uncertainty of the driver's future 
actions. Furthermore computational capacity is limited by the 
microprocessors used in automotive applications. In this paper con- 
siderations for dealing with the uncertainty of the estimated param- 
eters in the decision making process is discussed. Risk metrics and a 
computationally efficient method for decision making is proposed. It 
will be shown that under certain conditions the risk for a to early 
intervention can be kept constant for different closing velocities 
using the proposed method. Furthermore tracking and modelling of 
driver, sensors and brake system is discussed. The models of driver 
actions and of the radar sensor measurement error are based on 
measurements from a Collision Avoidance system equipped Volvo 
V70, provided by Volvo Car Corporation. 



1 Introduction 

A current trend in automotive industry is to introduce active safety systems 
that avoid or mitigate collisions. One system with a potential large positive 
impact on accident statistics is forward collision avoidance systems (FCAS), 
using sensors such as radar, lidar and cameras to monitor the region in front 
of the vehicle. A tracking algorithm is used to estimate the state of the objects 
ahead and a decision algorithm uses the estimated states to determine any 
action. In figure la schematic picture over a FCAS is presented. 

In this paper we will specifically discuss how to deal with the uncertainty of 
the estimated parameters in decision making process of a FCAS. Specifically a 
system that performs late braking to reduce the collision speed, which is 
referred to as collision mitigation by braking system (CMbB system), will be 
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considered. References to this type of system can be found in [3], There are 
several motivations for FCAS. One is their potential ability to affect rear-end 
collisions which constitute approximately 30% [6] of all collisions. Furthermore 
human factors contribute to approximately 90% of all traffic accidents [5]. For 
reasons such as driver acceptance and legal requirements of the system not to 
make faulty decisions, the tracking and decision algorithms are crucial. 




Fig. 1. Illustration of a forward collision avoidance system. This system 
warns the driver of a potential collision, by means of flashing lights 
reflected in the windshield. 



2 Tracking and Vehicle Modeling 

For a CMbB system the measurement noise (radar, lidar camera etc.) and the 
process noise (driver inputs) are not necessarily Gaussian. The process model 
of a car's motion is in general non-linear. A standard approach (which will be 
adopted in this paper) to deal with the nonlinearities is to make time varying 
linear approximation of the system and apply the time varying Kalman filter 
(this also requires the noise distributions to be approximated with Gaussian 
distributions). 



Dealing with Uncertainty in Automotive Collision Avoidance I 167 



2.1 Tracking Model 

The following state space model for the vehicle dynamics and sensor meas- 
urements will be used: 



For CMbB, the state vector, contciins relative position (longitudinal and 

lateral) to the other vehicle of potential collision risk, and relative velocity. 
Other quantities that might be estimated are relative acceleration and yaw- 
rate. Further, Wj. denotes the driver inputs from the accelerator, brake and 
steering wheel. The driver is thus modeled by the probability density distri- 
bution of the stochastic variable Since it also includes actions in the 
other car, has to be considered as un-measurable. The measurement relation 
(lb) comes from distance and bearing sensors such as radar, lidar and camera. 
The measurement noise, includes clutter, multi-path, multiple reflection 
points and other artifacts introduced in the low-level signal processing. The 
probability density of Cf, p/.) is used to model the sensor noise properties. If 
we only consider longitudinal motion we can use the following model 



Here tracking will be performed in a relative coordinate system, the states are 
relative position p , velocity v and acceleration a . The control signal a^^pis the 
requested acceleration of the host vehicle. A more thorough discussion on how 
to model the brake system follows below. The tracking model with sample 
time T is then 

= Axt -f Bu.tut -f HwW( (3a) 



— FfXf + B„tut + HwWf 
Vf = Btxt + e-f. 
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(lb) 
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T^/2\.h{xt) = {p) (4) 
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2.2 State Noise Model 

When using the EKF for target tracking, one assumes a Gaussian distribution 
for Wf. The main source of the state noise (w) is the driver. However the input 
from the driver is limited by the performance of the vehicle. Factors that play 
a significant role are available tire-to-road friction, brake- and steering-system 
characteristics and engine power. A vehicle's dynamical properties change 
with time and the driver's behavior also typically varies with road/traffic con- 
dition. We try to model all these phenomena with a Gaussian noise, clearly a 
very coarse model. To get realistic values on the process noise, data was col- 
lected during normal driving in rural traffic. A histogram of accelerations dur- 
ing this drive is displayed in figure 2. 



Empirical distribution of vehicle acceleration 




Fig. 2. Probability density (p^i^host)) acceleration of a vehicle driven in 
urban traffic for 45 minutes. 



2.3 Measurement Noise Model 

The measurement equation involves range and azimuth angle measurements 
to the vehicle ahead. Similarly to the above, the range error distribution can 
be modeled. A common model is to assume Gaussian noise. Figure 3 shows the 
histogram of radar range measurements. 
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The data collection was performed with the target vehicle (a Volvo V70) tow- 
ing the radar equipped vehicle at constant speed. 




Fig. 3. Probability density of radar measurement pjip) at constant speed. 

The measurement where performed with the target vehicle towing 
the radar host vehicle. To minimize range variation due to rope 
elasticity the test where performed on a test track with even pave- 
ment and cruise control to keep the speed constant. 



2.4 Brake System Model 

To calculate the collision speed the brake system characteristics need to be 
known. A complete brake system model is very complex. However looking at 
overall vehicle performance a simple yet accurate description can be obtained 
using a first order system for the acceleration. Figure 4 shows measurement 
data from 10 brake maneuvers with a Volvo V70 and a first order approxima- 
tion (according to (5)). 
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Measured and simulated braking system 




Fig. 4. Data from a hard braking maneuver, 110-0 km/h with a Volvo 
V70. 

The measurements were performed on warm and dry asphalt. For the simu- 
lated data a first order system (5) was used, ki = l and the input 1 1 m/s^ 
was chosen such that the system rise time is 0.3 s and the stationary value is 
11 m/s^. 






« + ki 









(5) 



Comparing traveled distance after 2 seconds we find that the difference is less 
than 1 m which is sufficient for our purposes. More information on typical 
brake system behavior can be found in [2]. 



3 Statistical Decision Making 

Statistical decision making requires an estimate of the distribution of from 
measurements A general model-based statistical decision criterion for the 
risk metric g(x) takes the form 
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Pr{g{xt) > gat] > 1 - a 



( 6 ) 



where a determines the confidence level. Note that this is a more flexihle 
approach than a rule like 



since in (6) we use the complete a posteriori distrihution of the state vector and 
not only the estimate x, . How successful is the rule (6) in mitigating a colli- 
sion? That depends of course on the rule g(x) itself, hut also on how accurate- 
ly the prohahility density function (PDF), p(x), reflects the true distrihution of 
the relative distance between the vehicles. Which rule should he used (6) or 
(7)? Both approaches have their pros and cons. While (6) has the advantage of 
a fixed confidence level, for making a decision, an analytical solution is in gen- 
eral hard to find. Thus a computationally expensive algorithm is often required 
to give a numerical approximation [4]. The rule (7) on the other hand is com- 
putationally cheap, but if p(x) varies a lot the probability for making the deci- 
sion might also vary significantly. In this paper we will study a third alterna- 
tive decision criterion. The aim is to keep computational complexity low while 
taking into account the time varying uncertainty of the decision quantity. We 
will use the following criterion. 



Here B(.) and D(.) denote bias and standard deviation respectively and Q are 
design parameters. One metivation for (8) is the well known statistical proper- 
ties of the Gaussian distribution. If g(Xi) is Gaussian, (8) corresponds to mak- 
ing the decision with a certain confidence i.e. the same as in (6). 

3.1 Estimating Statistics 



9{xt) > 9th 



(7) 



g(xt) - CiB{g(xt)) > ga, + C 2 D{g{xt)) 



( 8 ) 



How is B(g(xJ) and D(g(xJ) calculated? The bias and standard deviation of (8) 
is defined in the following way 
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Bias ; Dig{x)) = E{g{x)) - g{x) 



(9) 



Standard Deviation ; D(g(xi)) = \jE(g^(xi)) — E{g(xi))^ (10) 

where E() denetes the expectation value. E(^(xt)) is known as the 7 ^^ order 
zero point moment of g(x), which is defined by 



p(g(xj) is the PDF of g(xj. Considering that g(.) can be a quite complicated 
function the integral ( 11 ) or even the p(g(x^) might be hard to compute 
analytically and computationally costly to approximate by numerical integra- 
tion. Hence we will use an approximation of the expectation value in (11), 
where an assumption of a Gaussian distributed x^ is used. The approximation 
is obtained from the expectation value of a Thylor expansion of ^(xj given by 



where and x- denotes the element of the state vector x^ An 

approximation of the expectation E(^(x^) can thus be found from the expres- 
sion 



By choosing how many terms of the series expansion to consider the accuracy 
and computational burden can be traded off. If we neglect terms higher than 
the second order we of course get the well known Gauss approximation for- 
mula. To evaluate (13) it is clear that higher order moments of xy need to be 
known. In the case these can be calculated from the characteristic function 




( 11 ) 




( 12 ) 
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2 



Here h=[b_l b_2 ...b_n]^ and P is the covariance matrix of x. The order 
moments of x,- are given by 






dbi 



(15) 



Thus inserting the approximation from (13) into (9) and (10) an approximation 
of B(g(xJ) and D(g(xJ) can be calculated. The same reasoning can of course 
also be used to estimate other interesting statistical properties such as e.g. 
skewness and kurtosis. 



3.2 Application to Collision Avoidance 

In this paper we will study a CMbB system, which performs maximum brak- 
ing when a collision is becoming imminent. By imminent we mean that a 
maneuver that is close to or exceeds the vehicles handling limits is required to 
avoid collision. To simplify the analysis only longitudinal motion will be con- 
sidered, in general both lateral and longitudinal motions need to be consid- 
ered. A vehicle's handling properties are often measured in terms of accelera- 
tion. One appropriate risk metric for a CMbB system is to calculate the accel- 
eration required to avoid collision. Under the assumption that the tracked vehi- 
cle continues with constant acceleration the relative required acceleration is 
given by the solution to 

0 = p/ + Vito + (16) 



where tg is the time to accelerate to zero relative velocity and p , v , a are rel- 
ative position, velocity and acceleration respectively 







( 17 ) 
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The host required longitudinal acceleration will thus be given by 



^h,t 



!>.l __ 

^Pl 



( 18 ) 



Here ^ and ^ denotes host and tracked vehicle acceleration. The equation 
(18) was derived assuming that the maximum deceleration can be achieved 
momentaneously. Looking at figure 4 it is clear that this is not entirely true. A 
better risk metric also considers the brake system rise time. Using the model 
(5) for the brake system we get the following expression for the relative accel- 
eration (again we assume constant acceleration for the tracked vehicle) 



(i(l) ^Tcf 



(19) 



where the input requested acceleration. The velocity and traveled dis- 

tance during deceleration is given by 



v{t) = i-'o - + a„ft + - 






-kit 



( 20 ) 



^rt’f I ^’1‘f I I 



-kit 



A-2 



( 21 ) 



The time to bring the velocity to zero is given by the solution to (19) which 
yields 



qo^i-.i,,, f) 

(rpA-i - n,.,.f - L(-r “" f )a,., y) (22) 



where L(x) denotes the solution to we’"=x. Combining (22) and (21) allows the 
stopping distance to be calculated. The new risk metric will thus be the stop- 
ping distance calculated for this is compared to the measured dis- 

tance, to determine any action. We shall now investigate performance of a risk 
metric based on maximum deceleration when used as a decision criterion 
according to (7) and (8). To keep the analysis simple the brake system rise time 
will be neglected, and thus (18) will used be as risk metric. 
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4 Simulations 

To evaluate the different decision criteria, a scenario where a CMhB system 
equipped vehicle is approaching an object at constant velocity according to fig- 
ure 1, will be studied. This type of scenario corresponds to more than 30% of 
all rear-end collisions [1], First we will look at the probability of performing an 
intervention during the sample interval when the CMhB vehicle is exactly at 
the intervention boundary i.e. at the instant when a collision is becoming 
imminent. Here we define a collision as being imminent when a deceleration 
of 8 m/s^ or more is required to avoid colliding. In figure 5 the probability of 
an intervention for the criterions (7) and (8) can be compared at different clos- 
ing velocities, the parameters for the calculation can be found in table 1. 



Probability of intervention at decision boundary: Std.(p) = 0.21 , Std.(v) = 0.25 




Fig. 5. Probability of performing a CMbB intervention at the sampling 
interval when the collision imminent boundary is crossed. The 
required acceleration is -8 [m/s^] 

In figure 5 the probability of an intervention is displayed only for a certain 
time instant (i.e. when the collision is becoming imminent). Next we will study 
system performance in terms of collision speed and probability of making a too 
early intervention for the head-on scenario. A too early intervention is defined 
as a CMbB system induced brake maneuver before the collision imminent 
boundary is reached. The process noise, w, and measurement noise, e, are 
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assumed to be independent and Gaussian. The values of the noise variances 
were chosen to correspond to the measured pdfs of section 2. The brake sys- 
tem model is according to the model described in section 2.4, however the 
maximum deceleration is somewhat lower, since the perfect conditions on the 
test tracks are not always present on a normal road. Both decision criterion 
functions are applied to the tracking system output. A summary of the simu- 
lation parameters is given in table 1 . 



SI.Ml X.4 7 ION SI :.MMAR t . 



Decision criterion (7) parameters: g,» = -8 5 [in/s-]. 

Decision criterion (8) parameters: f.'j=l , Cj=l g*= -8 [m/s-]. 
Sensor and ttackii® system update rate: 1 0 Hz. 

Measmmient noise: <?, GjV(0,0.25) 

Brake system model: ki = 7 (rise time .tOO milliseconds). 



Table 1 . Summary of important simulation parameters. 

In figure 6 the simulation results are displayed for speeds from 5-60 km/h. For 
each speed 2000 Monte Carlo simulations were performed, and for each sce- 
nario both decision criteria were applied to the same tracking data. At higher 
speeds than 60 km/h a steering avoidance maneuver is much more efficient 
than a braking maneuver, therefore higher velocities are not as interesting to 
examine. 

Figure 7 displays the same simulations scenarios as figure 6, but here the stan- 
dard deviation of the measurement noise was doubled. 
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Fig. 6. Head-on scenario simulation results for Criterion (7) and (8) using 
the risk metric (18). The simulation parameters are summarized in 
Table 1 
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Fig. 7. Head-on scenario simulation results for Criterion (7) and (8) using 
the risk metric (18). Here the std. of the measurement noise was 
doubled (compared to figure 6). 



1 78 I Safety 



5 Analysis 



How well does the criterion (7) work compared to (8)? In figure 5 the proba- 
bility of an intervention at a certain time instant is compared for different 
velocities. From the figure it is clear that for the criterion (7) the probability 
for an intervention vary while for (8) it is kept constant (for speeds above 
15 km/h). Why is it that the decision criterion (8) manage to keep a constant 
risk level? To understand this we need to look at the criterion more closely. We 
rewrite the estimated required acceleration as 



^hofif ^obj 



■ ^obj 



2p 



= «nbj + - 

= “obj + - 



2(/7 -(- wp) 

(v^ + 2vwy + U^) 
2(;7 -I- u’p) 



(23) 

(24) 

(25) 



where in our case Wp, w^snd GN (0,o^. When p and v are large compared 

to the noise terms and (18) is well approximated by 



(^obj + 
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(26) 



Equation (26) has a Gaussian distribution since it consists of sum of Gaussian. 
Thus, when p and v are large compared to the uncertainty of the risk metric 
(18) we approximately have 



^hiosl 



+ 





4?- 
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2p 




(27) 



The criterion (8) will thus be equivalent to the hypothesis test (6), the param- 
eter C 2 will determine the confidence level of the test. What are the implica- 
tions of this in terms of collision mitigation performance? Looking at the sim- 
ulation results of figure 6, we see that for speeds of 30 km/h and above the 
probability of a faulty intervention is kept constant for the criterion (8). For cri- 
terion (7) on the other hand the probability of a faulty intervention decreases 
as the closing velocity becomes higher. At velocities around 35 km/h the per- 
formance is the same for both criteria. This is also the velocity at which they 
cross in figure 5. Both criterions performed bad at low speed. This is maiinly 
caused by the fact that the dynamics of the brake system is not captured in 
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the risk metric (18). The assumption of a constant measurement noise causes 
the assumption of p and v being large compared to the noise to be violated 
at low closing speeds. The required acceleration (18) will thus not be Gaussian. 
This causes the probability of intervention to vary significantly for low speeds. 



6 Conclusions 

In this paper statistical decision making applied to automotive collision avoid- 
ance has been discussed. A criterion that explicitly uses an approximation of 
the statistical properties of the risk metric has been simulated and analyzed. 
To compute the approximation the estimated tracking noise covariance (which 
was assumed to be Gaussian) was used. The main reason for this criterion was 
to be able to deal with changing uncertainty of the risk metric without having 
to evaluate the entire PDF. The criterion is motivated by properties of the 
Gaussian distribution, for Gaussian distributed variables the criterion corre- 
sponds to a hypothesis test. It was shown that one particular risk metric (18) 
can be considered to be Gaussian under certain circumstances. In head-on col- 
lision scenarios where the assumption of gaussianity holds (v > 25 km/h), the 
criterion (8) applied to (18) keeps the risk of a faulty intervention at a relative- 
ly constant level. However, a careful comparison of the results in figure 6 and 

7 shows a slight change in that risk level. For the criterion (7) the risk of a 
faulty intervention is changed more significantly as the measurement noise is 
increased. For scenarios where the assumptions do not hold (e.g. at low speeds) 
the criterion (8) becomes too cautious, causing braking to be too late. Bad per- 
formance at low speeds are also caused by the fact that the risk metric used 
does not account for the brake system dynamics, here the second risk metric 
discussed in Section 3 can offer significant improvements. 
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Abstract 

PAROTO is a joint project investigating obstacle detection and avoid- 
ance by fusion of information from radar and infrared imagery with 
the support of the French ministry of Research. The PAROTO con- 
sortium includes SAGEM, LASMEA, INRETS and TEAM. 

Detection of obstacles by radar and infrared exhibit different prop- 
erties so their concurrent use allows optimal performance. 
Innovative image processing algorithms have heen set up to detect 
vehicles and pedestrians (see [6]). Original 77 GHz radar was also 
specified and realised. Fusion of information exploits a sound math- 
ematical framework getting the best of both alarm sources. Then a 
filtering step eliminates pointless alarms, when it is clear that the 
driver has already taken appropriate actions regarding the obstacle. 
A dedicated driver's interface was also designed and realised to 
warn - visually and/or acoustically - without unnecessarily driving 
the driver's gaze out of the road ahead. 

Roth detection schemes already operate in real time. By the end of 
2003, the fusion of information, integrated aboard our tested vehi- 
cle, will be evaluated together with the driver interface, providing 
valuable insight into automated perception dedicated to automotive 
safety. Results strongly incite us to anticipate the design of a low 
cost hardware dedicated to these functions. 



1 Introduction 

The aim of PAROTO system is to compensate the lack of attention of the 
driver, a proven source of many of accidents. 
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Such a system depends on several mechanisms and assumptions: 

► A safety system should not rely on one sensor only. We have chosen two 
sensors: Radar and Infrared camera. They are sensitive to different 
physical properties of an object (materials and surface in one hand, ther- 
mal behaviour on the other). They also give complementary geometri- 
cal measures. 

► It must be discrete but reliable (through a dedicated HMI with gradu- 
ated warnings, depending on the situation). 

► It must be accepted by the driver to be efficient. 

► It must take into account the reaction of the driver facing the situation. 

In this article, we present each functionality of the PAROTO system. We first 
recall the functional architecture. We describe the Radar and the Infrared cam- 
era and their related processes dedicated to obstacles detection. Then, we 
focus on data fusion, which builds a precise map of obstacles ahead of the vehi- 
cle. We present the driver's interface and the filtering module. We conclude 
with a description of the system platform. 




Fig. 1 . VELAC, the vehicle of LASMEA, equipped with IR camera (red) and 
radar (green). 
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2 Functional Architecture 

Both sensors feed respective processing capabilities, which in turn interact 
with a fusion module, the output of which is filtered in order to send only rel- 
evant signals to the driver's interface (figure 2). 



Infrared Infrared 

Camera Processing 



Radar 



RADAR 

processing 



Fusion 



Filtering 



Driver’s 

Interface 



Fig. 2. Functional architecture. 



3 Radar Obstacle Detection and Tracking 
3.1 Radar Specifications 

The key interests to use a radar in this project are on the one hand the accu- 
racy of obstacles estimated speed and on the other hand the quality of radar 
data up to 1 50 m in spite of bad weather conditions. 

According to the regulation in the European Union and in the United States, 
the wave emission frequency for a car anti-collision project is fixed to 
fo=n GHz (millimetre wave). The principle of the radar used in this frame- 
work is a pulsed doppler radar with a Doppler priority. Indeed, the priority is 
given to the speed of obstacles with respect to the experimental vehicle. The 
distance of the obstacles is sorted by P range gates. 

Moreover, this radar has only one antenna for the emission and the reception 
which provide a 3. 3 “field of view. This system requires a circulator (ultra-high- 
frequency system) and a synchronisation process in order to transfer the ener- 
gy either between the transmitter and the antenna or between the antenna 
and the receiver (figure 3). The reception channel being very sensitive, the 
system must perfectly isolate this channel from the emission channel because 
of the strong emitted power. 
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Fig. 3. Structure of the PAROTO radar with one antenna. 

The radar technique used in this project consists in sending by repetitive puls- 
es (width r^=150 ns and period T^= 4 ps) a sinusoidal electromagnetic wave 
and to receive and analyse the signal reflected by a target (see figure 4). The 
data resolution is: 






ct^ 

~1 



3-10’^-1501Q-’ 

2 



22.5m 



( 1 ) 



dv = - 



3 10* 



2 ■ 77- 10" -4 •lO'*’- 2048 



• = 0.238m/s = 0.856km/h 



( 2 ) 



where N represents the period number (actually 2048 which correspond to 
8 ms of time analysis) of the pulse repetition . 



V 




Fig. 4. Emission and reception chart. 
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3.2 Obstacle Detection and Tracking 
Echo extraction [1 ] 

The process first acquires, each n x T/n G [0,....N-1]) time, an array of P x N 
(3) data corresponding to the sample of the two I, Q reflected signal (0°, 90°) 
provided by a high speed 40 MHz two lines converter stage. Then we compute 
an estimate of the power spectral density by FFT (Fast Fourier Ifansform) 
analysis. 

In this new array, we apply a threshold-clustering algorithm and extract 
echoes. 

An echo is then defined by four parameters: 

► time: 



hxT;(«g[0,...,A1-1]) (4) 

► amplitude: value of the power spectral density versus frequency 

► Doppler: position in the column's power spectral density array 



K, = I column - — I ■ (Iv 
2 



(5) 



► range: position in the line's power spectral density array 



r = {line-iy BD + 



W 

2 



( 6 ) 



Kalman filter tracking [2] 

A constant velocity model is used into a Kalman filter to track echoes. This 
tracking is performed into three steps: 

► Initialisation of tracks (r, VJ using coherence in many frames, 

► filtering of tracks: prediction and estimation, 

► data association. 

This tracking yields a range estimate more accurate than the gate value 
(22.5 m), and also obstacle speed, as it is shown on following results. 
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3.3 Some Results 

On the following illustrations we present some results of our tracking in foggy 
conditions. 



Tracking tesulls for range 






Figure (b) 



Fig. 5. Tracking results. 

On these illustrations, we show that the tracking algorithm provides a range 
estimate more accurately than a range gate. For the relative speed, we can see 
that the estimate corresponds to radar measurement. Furthermore, we can see 
that when obstacle's relative speed becomes positive the range increases. 
Finally as we can see, we are able to track an obstacle even if it is not visible 
(figure 5b). 
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3.4 Implementation 

In order to validate this approach in a real navigation context, this Radar has 
heen implemented on the experimental vehicle (VELAC) of the LASMEA 
Lahoratory (figure 6). 




Fig. 6. Radar location on VELAC. 



The acquisition of the analogical data from the Radar is carried out on a stan- 
dard PC equipped with an acquisition card AD-LINK PCIS-9812. 



4 Infrared Obstacle Detection and Tracking 

We first present the sensor technology and its location on the vehicle. Then we 
recall the infrared image processing. We conclude this section with some new 
results. 



4.1 Infrared Camera Specifications 

Some car manufacturers already propose, or at least evaluate infrared cam- 
eras. 

Our infrared sensor is an uncooled 8-12 micro bolometer. Its thermal resolu- 
tion is 80 mK. The size of images is 320 x 240 pixels. The optical design allows 
a 40° X 30° field of view. 

The camera is located on top of the windscreen, for practical and theoretical 
reasons (dirt, projections and geometry of distance estimation). 
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4.2 Infrared Image Processing 
What’s hot: 

In classical road situations, the sources of danger relate to human activity. This 
leads us to focus the detection on vehicles, cycles and pedestrians, which 
exhibit strong and typical behaviours in infrared. 



w-.' 




Fig. 7. IR aspects of road users. 

Detection and tracking: 

Detection of objects of interest is done in two steps: 

► Extraction of various cues from image, using low-level processing mod- 
ules. One of these modules extracts significant hot regions, which are 
typical products of infrared image processing. Other cues are based on 
geometrical criteria, gradients shapes, symmetries, 

► Merging of low-level cues into objects, assuming loose geometrical tem- 
plates of those objects. 

Then positions of detected objects are mapped onto the road, under the 
assumption that the road belongs to a plane. Since this operation is sensitive 
to the changes of pitch, motion analysis - based on feature tracking - is per- 
formed in order to estimate the pitch and correct the 3D positions according- 
ly. 

At this point, we track objects using a constant velocity Kalman filter. 

Some results: Vehicle detection 

On the following illustrations, black boxes are frame-by-frame detected vehi- 
cles since tracking can only be shown on live demonstration. The algorithm 
works in real time. We have experimented it in various situations, as suggest- 
ed below. 
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Fig. 8. Vehicle detection in a rural context. 




Fig. 9. Vehicle detection on highway. 
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Fig. 10. Vehicle detection in an urban context. 

Figure 10 shows results in an urban context. Complex situation implies many 
possible sources of false detection. 

Pedestrian Detection in Various Contexts 

The pedestrian detection process has not reached such a completion status as 
the vehicle detection one. However, evaluated on many sequences at hand, our 
low-level detection modules have performed quite well, as the following 
images illustrate it. 

Note that at this point, a same pedestrian may be detected in two parts, due 
to the still crude merging of our low-level cues. 
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Pedestrians In an simple context Pedestrian walking on our lane 




Urban context Rural context, oedestrlan crossing our lane 



Fig. 11. Pedestrian detection. 



5 Data Fusion 

Whereas radar is able to tell at what distance it points echoes, the direction 
being somewhat fuzzy. Infrared can give the direction in which an event is 
detected, but no precise access to the distance at which it occurs. Information 
coming from those sensors complements each other. 

Data from both sensors are first processed separately. Processes go as far as 
possible in the interpretation of the scene, to extract the most relevant infor- 
mation from sensors. Then the outputs of processes - the tracks of detected 
objects - are passed to the fusion module. 

From those tracks, the fusion module builds a map describing the situation 
ahead of the vehicle. Once built, this map is sent to the filtering module, the 
goal of which is to determine whether the driver should be alerted or not. 
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5.1 Areas of Fusion to Consider 

Radar and infrared cover complementarY areas ahead of the vehicle (figure 
12). We call a track or an area "dual mode" when both sensors are involved. 




Fig. 12. Areas covered by PAROTO system. 



Tfacks produced by single processes lie in one of the three areas described in 
figure 12. If an IR (resp. radar) track is produced in the IR (radar) exclusive 
area, the fusion module cannot confirm it with a track from the other sensor 
process: it must rely on this single sensor. Whereas, if the track is produced in 
the dual mode area, a real data fusion can be performed. 



5.2 Multiple Sensors, Multiple Target Tracking: a Hybrid Model of Fusion 

There are three main steps in fusion process. First, we must determine which 
tracks can be associated. For each couple of tracks, we improve the informa- 
tion concerning the object represented. Finally, we maintain tracks over time. 

Data Association 

Observations from IR and radar are dated with respect to a common clock, so 
they can be compared. When trying to associate an IR track with a radar track 
in the dual mode area, tracks are extrapolated at the current time and com- 
pared. 

The comparison is done on relative distance and speed, which compose the 
common space of measurements. 
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We assume that measurements are given with a Gaussian error. Then, we cal- 
culate a Mahalanohis distance between the IR track and the radar track and 
compare the result with a threshold to decide if the tracks refer to the same 
object. 

Data Consolidation 

A couple of IR and radar tracks matched by the association step are combined 
into a new dual mode track. A Kalman filter computes the position and veloc- 
ity of this dual mode track, which are given with better accuracy than those 
of its IR and Radar components. 



Track 




Fig. 13. Data consolidation, improvement of measures. 

Dual Mode Track Management 

A dual mode track can disappear in the following situations: 

► a sensor track (or both) disappear, 

► associated tracks do not represent the same object and diverge from 
each other. To avoid keeping such a dual mode track, we regularly check 
the fit of the sensor tracks. If it becomes too large, we delete the dual 
mode track. 

When a single sensor track popping in the dual mode area is not matched with 
any track from the other sensor, it is considered as a false alarm thus avoiding 
spurious detection from one sensor. 




1 94 I Safety 



5.3 Outputs of the Data Fusion 

We illustrate here some situations and show that fusion gives a significant gain 
in respect with separate sensor processing. 

Simple Situation 

Figure 14, we observe two vehicles in the image. After sensor processing step, 
IR and radar send to the fusion module the produced tracks. 



»Mi iirojmni IR ikHwiom Bfroi'e fusioti .\fter fuision 




Fig. 14. Data fusion on a simple situation. 

The errors on measures, symbolised by ellipsis, are as expected: IR tracks are 
less accurate in distance than radar tracks, and errors on IR measures grow 
with distance, whereas direction estimates are more accurate in IR. 

The vehicle positions propose different cases: 

► the vehicle on the left is in the IR exclusive area; its IR track cannot be 
confirmed by any radar track and is passed directly through fusion, 

► the vehicle on our lane is in the dual mode area: its IR and radar tracks 
are close enough and associated to build a dual mode track (blue in the 
right hand side map). 

Our observation confirms expectations: the error area of the dual mode track 
is far smaller than its single sensor counterparts. This is a mathematical con- 
sequence of the combination of two Gaussian distributions. 



Paroto Project: Infrared and Radar Data Fusion for Obstacle Avoidance I 195 



Overtaking of a Vehicle 
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Fig. 15. Overtaking of a vehicle, step 1. 
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Fig. 16. Overtaking of a vehicle, step 2. 

This case of overtaking illustrates the complementarity of infrared and radar. 
We approach the preceding vehicle (figure 15) and pull out (figure 16, at the 
beginning of the manoeuvre). With the radar alone, the system detects a vehi- 
cle ahead approaching until the last moment. Detection direction from 
infrared influences the dual mode track, allowing to detect the drift of the 
vehicle, therefore not classified as a threat. 

Note: although the vehicle seems to be detected out of the radar field in figure 
16, is due to the point wise representation, the radar actually detects the left 
side of the vehicle which is overlapping the dual mode area. 
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Security Barriers on Highway 

The vehicle is detected by infrared, and, being out of the dual mode area, does 
not require any radar confirmation. The radar detects the rail in the dual mode 
area, but IR does not confirm it, so it is discarded, therefore avoiding a false 
ebstacle. Symmetric situations (IR and no radar) also happen. 



Image wlih |)io)ec1ed IRdetmlon^ 
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Before fusion After fusion 





Fig. 17. Radar detection not confirmed by infrared. 



6 Driver’s Interface 
6.1 Visual Interface Design 

Constraints linked with the driving task have been defined to make the inter- 
face more effective: 

► Information given by the interface has to be detected witheut deviating 
the driver's gaze from the road and for that reason should be perceived 
in the driver's peripheral vision field. 

► Information given by the interface has to be detected at any time, even 
if the driver is looking in another direction or performing a secondary 
task. In that case, flashing lights could be better detected. 

► Information displayed has to indicate the obstacle approximate position 
quickly: blocks of LEDs could be dispatched all along the dashbeard in 
front of the driver. 

The visual interface is composed by 6 blocks (2 on the right, 2 on the left, 2 in 
the centre) displayed as follows: 
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Fig. 18. Visual interface. 



6.2 Warning Modes 

When it is required, signals are emitted through an interface to alert the driv- 
er. Two modes of warning can be displayed, mainly as a function of the time 
relationship between PAROTO car and a potential collision with the obstacle. 

Attention Orienting Mode 

When the situation is not of extreme urgency, the system will merely orient 
the driver visual attention to the obstacle direction. A set of amber flashing 
lights provided by LEDs will be displayed dynamically on the dashboard, in 
order to suggest the obstacle location and trajectory. In that case, the driver 
has time enough to take information from the environment and then, to opti- 
mize his or her response to a given situation. Such a warning makes possible 
to not disturb the driver with an intrusive alarm, when enough time is avail- 
able. Moreover, it could increase the driver's confidence in the system, by 
allowing him to compare his perception of the driving situation to what the 
system is showing. It could also act in such a way that the driver becomes 
familiar with the system's information presentation. This could lead to a bet- 
ter understanding of the displayed informatien and then to quicker reaction in 
case of critical situations, which are very infrequent. 

Alarm Mode 

When the situation requires an urgent response from the driver, the alarm 
mode is activated. This mode, which is more intrusive, combines two types of 
displays: a visual display (red flashing LEDs located right in front of the driv- 
er) and an audio interface which produces sound. The alarm mode may be put 
into operation if the situation is very urgent or following an attention-orient- 
ing mode if the driver has not reacted. 
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6.3 Filtering 

Various sensors on vehicle controls are also used to get informatien about what 
the driver is doing. Is he or she braking, turning the steering wheel, ...? These 
sensers then inferm the system whether er not the driver has seen an obsta- 
cle. For example, if the driver is well aware of the presence ef an obstacle en 
the road and had already apprepriately modified his er her behavieur te take 
it into account, it is not useful te provide him or her any alarm, due to the fact 
that alerting in this case, ceuld lead to a decrease in the system acceptability. 
Designed using rule-based algorithms, the filtering medule takes both fusion 
and sensor data into account. 



7 PAROTO Platform 

Infrared, radar and fusion modules run now in real time. Considering compu- 
tational power needed by infrared and radar processing, we selected a dis- 
tributed architecture, with ene PC for each main task. Computers, running 
under different operating systems (which choice depends on specific needs), 
are connected te an IEEE 1394 bus and a standard LAN, thus allowing large 
band pass transfers. 

Mereover, filtering medule runs on fusion PC, making this computer connect- 
ed te the CAN bus, on which circulates data frem proprioceptive sensers. The 
filtering medule centrols the driver HMI through an RS-232 connection. 




Fig. 19. PAROTO architecture. 
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8 Conclusion 

We have presented the obstacle detection and avoidance function of the PARO- 
TO system. 

An infrared camera and a radar yield detection tracks, passed to a fusion algo- 
rithm, which builds a precise description of the situation ahead of the vehicle. 

Infrared, radar detection and fusion run in real-time on the testbed vehicle 
belonging to LASMEA This ambitious detection schemes has been thoroughly 
tested and proves very efficient in many different contexts. 

The detected tracks are then passed on to a filtering process, which in turn 
decides whether it is pertinent to alert the driver, and the most appropriate 
way to do it, through a dedicated interface. 

All these processing steps have been integrated successfully on the testbed 
vehicle, and are about ready for a thorough evaluation by INRETS experts in 
the coming weeks. 

The major interest of all the technical achievements throughout the PAROTO 
project pushes us to investigate a low-cost industrial implementation. 
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From Passive to Active Safety Systems 
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Abstract 

State of the art automotive restraint systems have reached a very 
high level of occupant protection. Vehicle construction, belt and 
airbag systems are the most important lifesavers in this respect. The 
vehicle installation rate of restraint systems covers almost 100%. 
The development of passive safety systems is focused on an 
improved performance of the restraint devices while the need for 
the protection of all road traffic participants is increasing. This can 
be seen in upcoming regulations like pedestrian protection systems 
in the EU and the increasing activities in the area of active safety 
like TPMS in the US. This requires new types of sensors and system 
layouts. New technologies, material and processes will help to design 
and build new sensors with higher reliability and increased func- 
tionality. 



1 Vehicle Safety Systems 

Passive restraint systems have become standard equipment since the late 80'. 
We can find these systems in all vehicles that are on the road today. The sys- 
tems are known as airbag systems and are state of the art of passive safety sys- 
tems. Additional protection devices like foot bags, knee bags, inflatable belts 
and active headrest have also been introduced over the last years. All these 
systems have in common that the decision for activation of the restraint 
devices such as pretensioners, front airbags, side or head airbag is based on 
the signals from acceleration sensors. The accelerometers are installed in the 
airbag control unit and also in the side and upfront sensors that are used to 
sense the impact as early as possible. 

The development of the restraint devices is following the general trend to 
more individually controlled devices as it can be seen on airbag modules. First 
systems used single stage inflators, the next generation was equipped with 
dual stage devices and latest developments are based on multi stage inflator 
technologies such as variable output inflators. 
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With the introduction of the different types of airbag systems also different 
type of crash sensors are required. For the first airbag systems with driver 
only or driver and passenger airbags a central crash sensing was required. The 
electronic modules where equipped with a single accelerometer or in cases 
where also an offset performance was required, two accelerometers in a ±45° 
orientation relative to the x-axis of the vehicle have been installed. This is the 
most common arrangement, which is still used in today's airbag control mod- 
ules. 

With the introduction of side airbags and head protection systems like 
Inflatable Curtains or Inflatable TUbular Structure additional side sensors were 
required. The first side sensors used an accelerometer and a micro controller 
for signal pre processing. The severity of the impact was transmitted as a 
PWM signal to the airbag control unit. Today's side sensor elements have a dig- 
ital communication, transmitting the actual crash signal to the airbag control 
unit, where the signal is processed in the crash algorithm. 

Up-Front sensors represent the latest level of innovation of a passive restraint 
system. They are installed close to the vehicle front and are used to detect a 
crash signal in the earliest possible phase of the impact. The purpose is a bet- 
ter discrimination of certain crash scenarios as well as to classify the severity 
of the crash to improve activation of smart airbag systems. 

Some vehicle types like off road vehicles or SUV's use additional roll over sen- 
sors. The required components, gyros and low-g z-sensors are integrated into 
the airbag control module. 



2 Crash Sensors 

Early airbag systems used mechanical crash sensors. The sensors have been 
mounted on structural elements in the frontal area of the vehicle. The electri- 
cal contacts of this sensor type were closed on an impact and the airbags are 
inflated. A discrimination of the crash pulse was not possible with this type of 
sensor. 

The next big step in the development of airbag control modules was the intro- 
duction of so called single point sensing systems. This type of systems became 
possible with the introduction of electronic crash sensors. The early genera- 
tion of sensors have been piezoelectric sensors, followed by bulk micro 
machined resistive sensors and surface micro machined capacitive sensors. An 
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improved discrimination of the crash pulse became possible with the develop- 
ment and introduction of this type of sensors. 

The state of the art crash sensors consists mostly of a micro machined sensing 
element and an ASIC that is used for signal conditioning, amplification and fil- 
tering. The ASIC also includes an interface that serves as a connection to the 
control module. The sensor is in most cases supplied with the required oper- 
ating voltage by the control module and the signals from the sensor to the con- 
trol module are transmitted by current modulation. Figure 1 shows a typical 
two chip crash sensor assembly and the block diagram. 




Self Test Status 



Fig. 1. Sensor assembly. 

When we look at the future of the safety systems that we know today, the pas- 
sive safety systems, we have to realize that the penetration of passive airbag 
systems in the market has reached a relative saturation. The major potential 
can be seen in the further development of smart ciirbag systems. Also a shift 
from passive safety towards active safety can be seen on the market. This is 
obvious with the increased number of systems that are used for improved 
braking like ABS, BAS or vehicle dynamics control systems like ESP and roll 
over prevention systems. A further development towards active safety sys- 
tems are advanced ACC systems. Stop & Roll and Stop & Go systems. 
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Future active safety system applications are e.g. lane detection, lane keeping 
support, driver warning and driver assistance systems, blind spot detection 
systems, crash avoidance and automatic cruising systems, which we might see 
in the future. 



3 Future Safety Systems 

From the actual road traffic and accident statistics it is known that passive 
safety systems have proven be very efficient in the reduction of injuries in 
vehicle crashes. Statistics it is also shows that there are cases where it is nec- 
essary to improve the performance of passive safety systems. 

This data also shows that protection for other road traffic participants is 
required, as there are motorbike drivers, bike riders and a large group of 
pedestrians. This can be seen in the regulation efforts regarding pedestrian 
protection systems of the European Union and the self-commitment of the 
European Automobile Manufacturer Association ACEA. 




Fig. 2. Market trends. 

Passive safety systems are activated at the moment when the impact occurs. 
The purpose of passive safety systems is to offer the best possible protection 
against injuries for the vehicle occupants, to limit the physical load of the occu- 
pant. Due to this limitation a new generation of sensor and sensing systems 
becomes necessary to fulfil the requirements of a future passive safety system. 

For a future safety system it is required to get information about a possible 
impact before the actual impact occurs. Depending on the range of the sensor. 
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the information is used for driver warning, for driver assistance and finally for 
crash avoidance. The signals will also be used for activation of a restraint sys- 
tem with a much higher precision and a better restraint performance com- 
pared to today's systems. To generate reliable information before the possible 
impact, it is required that the sensor measures the proximity of objects with- 
in the complete area of a vehicle. 



4 Pre-Crash Sensor 

A first generation of a pre-crash sensor consists of an optical sensor system 
that is mounted behind the windscreen, in a central position close to the rear 
mirror in the wiper area. The sensor principle is based on the speed of light. 
Infrared coded laser light is sent out and reflected from objects in the proxim- 
ity of the sensor. The decoder circuit calculates the time of flight between the 
emitted and received laser pulse. A wide beam arrangement has been select- 
ed for the sensor system to cover the complete area in front of the vehicle. 

• Pre-Crash Sensor 




I principle 
1 3 beams 
I wave length 
I range 
I accuracy 



laser pulsed time of flight 
15° wide, 8° 

910 nm 
<30 ft 
4" 



I velocity range 3 - 120 mph 



Fig. 3. CV sensor. 

The primary application of the CV Sensor is the usage as a pre-crash sensor. 
The sensor system measures the distance of an object in the front of the vehi- 
cle. The speed of an approaching object is calculated from the distance infor- 
mation of repetitive measurements. With the wide beam and multiple receiv- 
er arrangement it also possible to calculate the direction of the approaching 
object. 
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With this information it is very early known, before the actual impact occurs, 
which restraint device sheuld be activated. It is alse pessible to estimate which 
severeness the impact will have since the relative impact speed is calculated 
frem the sensor signals. This information is used to control restraint devices 
like adaptive force limiters er adaptive airbag systems as well as reversible 
restraint devices. 

Due to the fact that the relative impact speed can be calculated very precise- 
ly, the activation of restraint devices can also be staged much more efficient. 

A further advantage of the sensor arrangement is that the information is inde- 
pendent frem the vehicle structure, variations or ageing effects that might 
influence the crash perfermance of a conventional sensing system are elimi- 
nated. 

The key compenent for the pre-crash senser is a custom designed laser diede 
that censists of a laser chip the necessary pewer switch, a MOSFET, and the 
energy reserve which consists of two ceramic capaciters. 




Fig. 4. Circuit diagram. 

This optoelectronic micro system is a lead frame assembly. The cempenents 
fer the energy reserve are seldered to the lead frame, the laser chip and the 
MOSFET are attached as single chips. The connections are conventional bond 
technology, the complete device is moulded in a clear plastic material. 
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Fig. 5. Circuit diagram. 

A further application of the pre-crash sensor is to use it for an active pedestri- 
an protection system. The introduction in Europe is planned from September 
2005 on. The first systems will include an active hood, which is lifted for a cou- 
ple of centimetres to give additional deformation area. The next generation, 
that are introduced from 2010 on will include airbags at the a-pillar or the 
lower windscreen frame to protect the pedestrians from direct contacts with 
hard points of the vehicles. 

The purpose of the sensor is to detect an object, in this case a pedestrian in the 
vicinity of the vehicle and to predict the impact time and speed of the object. 
In parallel a measurement window is opened to receive a confirmation of the 
physical impact from an additional force sensor or bumper mounted 
accelerometers. With its capability to track objects after the confirmed physi- 
cal impact, "hard" and "soft" objects can be discriminated. This additional 
information is used for discriminate a pedestrian impact from an ordinary 
crash. The algorithm that is operated in the control unit will activate the nec- 
essary protection devices. 
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• Protection Devices 



■ active hood 

■ airbags at fire wallAvind screen 

■ airbags at a-pillars 

■ reversible restraint devices 

■ others 




• Safety Criterias 




physical impact 



■ two independent signals 

■ different technologies 

■ discimination crash vs. pedestrian impact 



Fig. 6. Pedestrian protection system. 



5 Active/ Passive Safety System 

The trend towards improved passive safety systems puts also new demands to 
the design of restraint systems. Due to the increased number of sensors the 
requirement to build up sensor clusters is obvious. Another important issue is 
the wiring effort for this type of systems. It has to be reduced for costs, assem- 
bly, weight, and quality and reliability reasons. 

Bus systems will be used for the connection and transmission of information 
between the different modules. There will be different types of bus systems, 
depending on the different applications that we can find for future restraint 
systems in vehicles. There will be high sensor bus systems as they are 
required for transmitting crash sensor information and there will also be sen- 
sor bus systems for smart peripheral devices. This bus system is used to trans- 
mit the information from buckle switches or seat track sensors to a restraint 
control module as well as to transmit information to actuators, e.g. reversible 
pretensioners. The Bosch/Siemens/Temic Sensor and Deployment Bus describe 
a capable bus system. Related bus compatible igniters for actuation of passive 
restraint devices are developed in the cooperation project IBA. 
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6 Conclusion 

Future restraint systems require new types of sensors and also new technolo- 
gies for the sensors. They will require pre-crash sensors with different sensing 
ranges, resolutions, wide and narrow focused beams. Their will also no longer 
be single sensor arrangements for future restraint systems, pre-crash sensors 
will also help to increase safety for all road traffic participants significantly. 
The challenge for the future is to find the best possible combination of the dif- 
ferent available sensors and technologies to get a maximum of performance at 
economical costs. 
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Abstract 

A 30 pg/sqrt(Hz) resolution MEMS based low-g 3-axis accelerome- 
ters targeted for emerging automotive application is presented. The 
device is composed by 2-dice in a single package: the sensing ele- 
ment and the processing IC. The sensing element is realized with an 
epitaxial micro-machining process (ThELMA™) while the IC uses a 
0.5 pm BICMOS process. The IC amplifier is able to detect capacitive 
changes as low as 10 aF (10 • 10'^® F) and the device is able to provide 
both an analog and a digital output. Moreover it has very good lin- 
earity and very low cross talk between the sensing axes thanks to a 
design that fits very well the ST micro-machining process ThELMA. 
Finally, the proposed interface accelerometer is factory-trimmed to 
ensure repeatable performance without production-line adjustments 
in the end product 

The device is intended for new inertial applications in the automo- 
tive field that are expected to be heavily dominated by micro- 
machined silicon components More specifically multiaxis accelerom- 
eters are particularly useful for emerging application like new gen- 
eration ESP and ABS system, assisted navigation, headlight leveling, 
car alarm and virtual horizon. 



1 Introduction 

LIS3L02 is the first 3 axis low-g linear accelerometer on the market able to 
measure 30 millionths (micro) of gravity acceleration with the lowest power 
consumption at 2.7 Volt supply integrated in a single package. The output data 
are available in analog and digital format. The device is also very robust 
against mechanical shock being compliant with drop test requirements. 
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LIS3L02 is a dual silicon chip solution inside a single package (PS024: 
15.3 mm x 7 mm x 2.5 mm or QFPN44: 7 mm x 7 mm x 1.8 mm): the sensor 
chip and the interface electronics. This approach allows the maximum flexi- 
bility, genericity and modularity together with a shorter development time for 
new products. 

Most advanced solution currently available on the market can provide only: 

► Maximum of 2-axis sensing feature integrated with the electronics in a 
single package. 

► 3 axis sensing element without the interface electronics. 

► Bulky solutions at PCB level realized by assembling 1 or 2 axis 
accelerometers. 

Among all the competing 3 axis accelerometers only ST 3-axis accelerometer 
is able to provide outstanding performances in a small package with low power 
consumption. 

The achievement of this breakthrough in the field of micro-machined 
accelerometers has been possible thanks to different innovations, which 
allowed overcoming the following technical challenges: 

► Achievement of an excellent process control in the manufacturing of an 
in-plane (XY) and out-of-plane (Z) sensitive micro-mechanical structure 
on the same die. The challenge was to optimize and control both the gap 
and the thickness of the sensitive element. 

► Development of two processing steps (Epitaxial Growth and Grinding 
technology) able to realize thick (15 pm) structures with low mechani- 
cal stress. This innovation step allowed reaching high resolution and 
high sensitivity without using the expensive Silicon-On-Insulator wafer 
and the difficult to industrialize bulk micro-machining technology. 

► The high robustness of the device has been obtained through an inno- 
vative mechanical design and the development of a high yield strength 
epitaxial poly-silicon layer (~ 5 GPa). 

► IWo level polysilicon layers and the high aspect ratio silicon etching 
technology allows to measure accelerations along the desired axis with 
low cross talk among the different axis. 

► Thinning of the silicon die and stacking of the sensing element and IC 
allowed the realization of such small package format 

► An innovative electronic circuitry architecture together with an accu- 
rate layout resulted in an excellent resolution for all the 3 axis with the 
small voltage supply and the low power consumption (less than 2 mW). 




A Low-g 3 Axis Accelerometer for Emerging Automotive Applications I 213 



2 Fabrication Process: ThELMA™ Technoiogy 

STMicroelectronics has developed a technological process for the realization of 
a complete family of inertial sensors (angular, linear accelerometers and gyro- 
scopes), This process is called ThELMA and it stands for Thick Epitaxial Layer 
for Microactuators and Accelerometers (see figurel). On a standard silicon sub- 
strate a thermal oxide is grown. On the top of the oxide a thin polysilicon layer 
is LPCVD deposited in order to realize buried interconnections. Afterwards a 
sacrificial oxide layer is deposed on the top of which a thick polysilicon epi- 
thaxy is grown (15 mm). This epithaxial polysilicon is the structural material 
that composes the sensor. The structure of the sensor is defined by deep sili- 
con RIE. At the end the sacrificial oxide is removed by using a vapor phase HE 
etching. 





L “ 




H Si substrate 
■ First oxide 
n Epitaxial poly 
r~l Metal 




Fig. 1. ST ThELMA micro-machining: 



■ Doped polysilicon 

■ Sacrificial oxide 
— Polyseed 




Schematic process description and 



scanning electron microscope pictures. 



With the process described above the sensor results to be thicker than sensors 
obtciined with thin film deposition techniques and therefore shows an higher 
sensitivity; moreover with respect to the structures obtained with surface 
micromachining technique it has greater stiffness to bending along z-axis and 
therefore a lower probability to stick on the substrate during sacrificial oxide 
removal. This process has also a lower cost than processes based on the use of 
SOI or silicon fusion bonded substrates. 



Due to the necessity to protect the devices during wafer sawing process and 
to guarantee a controlled atmosphere during their lifetime, a process for 
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encapsulating the sensors at wafer level has been developed. The process con- 
sists of using another silicon wafer to create the caps for the sensing structure 
with holes trough it in correspondence to pad region and of using an hermet- 
ic wafer to wafer bonding technique to seal the device as shown in the figures 
below. 



3 Chip Partitioning and Interface Electronics 

The device is composed by a triaxial sensing element and by an electronic 
front-end. which has been fully integrated in a 0.5 pm standard CMOS tech- 
nology. The overall system exhibits a 30/sqrt(Hz) • pg noise level and a full scale 
of 2 g or 6 g that can be selected by supplying the right voltage to a specified 
pin of the component. The front-end is directly connected on package to a tri- 
axial linear capacitive acceleration-sensing element and it supplies an analog 
voltage (V/g) or digital (SPl) output proportional to the input acceleration. The 
IC has the capability of processing signal coming from a sensor not realized in 
the same wafer, but connected in the same package. This feature allows the 
optimization of the sensor and the electronic front-end performance since they 
can be realized in different, and, then, customized and more suitable tech- 
nologies. The use of a single technology typically results in stringent trade-off 
in sensor and electronics performance. On the other hand, the cost of the 
choice of using different technologies is the larger parasitic input capacitance, 
which requires higher power consumption to reach the same dynamic range. 




Fig. 2. Encapsulated Sensing element and IC are shown on the left; the 
complete PS024 package (15 mm x 7 mm x 2.5 mm) and QFPN44 
(7 mm X 7 mm x 1.8 mm) on the right. 

The overall scheme of the proposed analog tri-axial linear accelerometer is 
shown more in detail in figure 3. A capacitive MEMS-based linear acceleration 
sensor is mounted, where a single mass is available to sense both x-axis, y-axis 
and z-axis acceleration. 
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The three sensing axes are intrinsicallY aligned precisely at 90° to each other. 
As a result, the board mounting is much simpler than when two separate 
2-axis linear sensors are used. In the sensor structure, a moving mass is pres- 
ent, while two stators for each axis are used and •^2x for the x-axis, and 
and for the y-axis, and % for the z-axis). 



66KHz 




Fig. 3. Sensing element in red block and IC on the right side of the picture. 

In case of the digital output device a proper ADC is present as an 
additional block of the above picture. 

The true sensor capacitances (whose variations have to be measured) are Csj^ 
and Cs 2 x for the x-axis, Csj^ and for the y-axis and Csj^ and for the 
z-axis. When a linear acceleration is applied to the sensor, the proof mass dis- 
places from its nominal position, causing an imbalance in the sensor capacitive 
half-bridges of and C$ 2 x for the x-axis, and of Csjy^^ and Cs 2 y^^ for the 
y,z-axis. The interface chip translates these minimal capacitance changes into 
calibrated analog voltages at the output pin proportional to the proof mass 
movement, and so to the applied acceleration. 

The capacitive imbalance is measured using charge integration in response to 
a fixed voltage pulse applied to the sense capacitors. In this way the interface 
front-end structure can be implemented by using differential switched-capac- 
itor techniques, which have demonstrated to be able to perform high dynam- 
ic range analog circuits. In addition, the complete signal processing uses a fully 
differential structure to improve system performance robustness. The multi- 
plex (MUX) and the demultiplex (DEMUX) techniques in the scheme are imple- 
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merited in the digital part controlling the switch and not using additional 
series switch which could he detrimental for the system performance. The off- 
set and I//" noise of the 1C are cancelled through the only-passive Correlated- 
Douhle-Sampling (CDS) structure implemented at its output nodes. This solu- 
tion has heen preferred to other ones implemented at the opamp input nodes 
due to its higher performance robustness with respect to parasitic capacitance 
and charge injection. 

As a final feature, a self-test mode is available, which is based on electrostatic 
actuation of the acceleration sensor proof-mass. Through a dedicated self-test 
pin is possible to electrostatically deflect the sensor beam at any time to veri- 
fy that the sensor and its electronic interface are functioning correctly. The 
sensor mobility and the electronic front-end can be checked using the self-test 
mode. When the self-test is selected, an actuation force is applied to the sen- 
sor simulating an input force. This produces a change in the DC level of the 
output of the device. 

Moreover the 1C has some poly-silicon fuses to trim the offset and the sensi- 
tivity of the sensors coming out of the fabrication line.ln case of the digital out- 
put device very low power Sigma-Delta converters are implemented for the 
three measurement channels, while keeping constant the overall resolution 
and power consumption thanks to smart tricks in the 1C design. 



4 Measurements Results 

4.1 Sensing Element Transfer Function 

To achieve the high performances of 3-axis accelerometer is is extremely 
important to control very well the mechanical parameters of the sensing ele- 
ment: the resonance frequency and the quality factor. The sensing masses 
have resonances frequeny between 2 and 4 kHz while the damping factor 
ranges from 0.5 for z-sensing element to 2 for xy-sensing element. Here below 
the reader can find the dynamic transfer function of the sensing element: 
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Fig. 4. Mechanical trasnsfer function of the z-axis sensing element and 
x/y sensing elements. 



4.2 Linearity of the 3-Axis Accelerometer 

Testing was performed in 5 different temperatures using a temperature cham- 
ber: + 85°C, -i-55°C, +20° -25°C and -40°C. The units were turned 90° in 10° 
steps starting from the angle where the gravity effect to the sensor was 0, 
using a rotary table. After 90 ° turn the gravity effect to the sensor was maxi- 
mum (1 g). 



The output voltage can be presented as a function of the normal acceleration. 
A best fit linear trend line is fitted to the results and error from the measured 
results to the fitted line is calculated. The result is average percentage part of 
1 g scale. All the results are definitely below 1 % of the product specs. 
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1 Temperature (H') 


Non-linearity (^o) I 


+85 


0,40 


+55 


0,25 


+ 20 


0,10 


+20 


0,32 


-25 


0,13 


-40 


0,18 



Table 1 . Linearity of the 3-Axis Accelerometer. 



5 Temperature Measurements 
5.1 Sensitivity Stability 

Sensitivity Stability over temperature has been measured and the results show 
that the sensitiviy has a thermal coefficient of less than 0.02 %/degree in the 
temperature range from -40 °C to -i-85°C. 



ST2 Sensitivitiy vs. temperature 
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Fig. 5. Thermal stability of the accelerometer sensitivity. 
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5.2 Offset Stability 

The package of our device is full molded and therefore the zero-g offset is more 
sensitive to the temperature effect. However besides the measurements 
reported below we are studying new mechanical designs and a new 1C aimed 
to improve the thermal offset stability of the 3-axis accelerometer to match the 
strong requirements of some specific automotive applications. 




•60 >40 .20 0 20 40 60 80 100 120 

Temp [C] 



Fig. 6. Thermal stability of the zero-g offset. The device is within ±10%. 



5.3 Cross Axis Sensitivity 

The perfect alignment guaranteed by the photo-masking step together with a 
good mechanical design allow to optimize each sensing element for each sen- 
sitive axis. Therefore the cross axis sensitivity is lower than 1 % while products 
based on bulk micro-machining are able to guarantee only 4-5% of cross axis 
sensitivity. This feature is extremely important for all the applications where 
the spurious accelerations must be rejected. 



5.4 Complete Measurement 

The following graphs show the analog output and digital output of the 3-axis 
accelerometer from ST Microelectronics. 
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I X axis Y axis Z axis | 





Fig. 7. Analog and digital output graphs of 3-axis accelerometer. While 
the first graph show results in V/g the second has a number asso- 
ciated with the input acceleration. 



6 Conclusions 

We have developed a low power consumption (<2 mW), high sensitive, 3-axis 
accelerometers that could be used for many emerging applications in the car. 

The device has an analog and a digital output and a bandwidth quite large. 
The stability of the sensitivity over the temperature range is very good togeth- 
er with the linearity and the cross axis sensitivity (< 1%). 

Some features need still to be improved for some automotive applications, but 
the first results currently available on new mechanical structures in different 
packages show very good promises for further improvements of the devices 
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while always keeping the power consumption very limited and the sensitivity 
very good. 
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Abstract 

The development of new powertrain functions has many goals: more 
power and torque, low fuel consumption and emission, best driving 
performance and a brand specific acoustics. In the future the accu- 
racy of control in Motor Control Units (MCU) will increase more and 
more. For this reason open loop control will replaced by closed loop 
control. New and/or other values are needed for better algorithms in 
powertrain control. Autonomous sensor systems for automobile 
industry offer increased reliability due to the reduction of cable con- 
nections, novel sensing applications made possible by wireless data 
transfer and cost advantages resulting from the aforementioned 
aspects. These systems consist of a power generator, energy storage 
and data transceiver. The technological challenge for realizing such 
system is most of all the construction and fabrication of a mini power 
generator that matches the dimensions of standard sensor modules. 
In this paper different power generators for autonomous sensor sys- 
tems are reviewed and systematically discussed. It is shown that low 
power thermoelectric generators can supply autonomous sensor sys- 
tems with up to 7 mW utilizing waste energy of the engine. 



1 Power Generation in Autonomous Sensor Systems 

Condition monitoring systems in nowadays cars apply more than 120 sensors 
distributed throughout the body of the car. Wires to connect them impose 
many limitations, especially if the sensor is located in a harsh environment or 
a remote location. Sensor-to-controller cables bring disadvantages to the cost 
structure, the weight and therefore emissions of the automobile and the relia- 
bility of the sensor system. Autonomous sensor systems promise a new gen- 
eration of wireless sensor networks for automotive industry. Autonomous sen- 
sor systems will reduce the number of cable connections, will make novel sens- 
ing applications possible by wireless data transfer and will result in valuable 
cost advantages. 
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A wireless sensor system consists of a power generator, energy storage and 
data transceiver (figure 1). 




Fig. 1. Graphical illustration of a wireless sensor, possible energy sources 
and its communication with external control units. 



The technological challenge for realizing such system is most of all the con- 
struction and fabrication of a power generator that is small enough to match 
the dimensions of standard sensor modules. Micro- and Nanotechnology are 
privileged to solve such challenges and power generators with output of sev- 
eral microwatts were presented in literature [1-3]. Earlier work on piezoelec- 
tric, electrostatic and electromagnetic vibrational energy conversion schemes 
illustrate the great interest in micro power generators for applications where 
batteries are unacceptable [5-7]. Looking at the energy budget of an automo- 
bile someone finds that only roughly 25% of the energy are used for mobility 
and accessories. Approximately 75% of the energy supplied for combustion 
are lost as exhaust gases, coolants, friction and radiated energy. The conver- 
sion of thermal and kinetic energy into electricity is consequently the pre- 
ferred principle for energy supply in autonomous sensor systems for car appli- 
cations [4]. Kinetic energy conversion generated from vibrations was 
described in detail by Hettich et. al. [4]. The vibration response of the chassis 
of a Mercedes-Benz M-Class was measured for different road conditions. The 
typical peak acceleration, for example for driving on paving stones (30 km/h) 
is 40 m/s^. An inductive power generator as demonstrated by Hettich et. al. 
supplied a peak power of 10 mW whereas the average power of the generator 
is almost one order lower. The typical power range of automotive sensors is 
several milliwatt power, which is significantly more than what many of the 
above mentioned demonstrators are designed for. The energy density of 
usable kinetic energy compared to the energy density of heat in a car is about 
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two orders lower [7], Especially sensor applications in the powertrain and the 
exhaust system could greatly benefit form thermoelectric power generators. 

For data transmission with a radio frequency interface we use well known 
modules from commercial providers. Different carrier frequencies and proto- 
cols such as a bluetooth module, WLAN module or a 433 MHz or 866 MHz 
transceiver are available. Since they are described by suppliers they are not 
part of this paper. The choice of the appropriate transmitter depends on the 
transmitted data rate, interfaces and environment. 

For example for temperature sensors in automobiles a low data rate readout is 
sufficient since they do not change the value relative often. In many applica- 
tions it is not necessary to read the sensor output more then one time per sec- 
ond. A RF data transmission takes only a few milliseconds. The remaining 
time, the electronic is in sleep mode and does not consumes energy. For this 
kind of sensors, without high timing requirements, we can use advantageous- 
ly autonomous sensor systems with power generation. 



2 The Thermoelectric Generator 

This paper describes a new approach for power generation from thermal ener- 
gy in the powertrain of a car. A low power thermoelectric generator (LPTG) 
was developed and realized in cooperation with D.T.S. GmbH, Halle. The LPTG 
was manufactured with thin film technology methods and equipment for 
batch processes for film generation, photolithography, pattern generation, 
wafer and foil cutting and device assembly [8]. This approach is different from 
standard thick film technologies offered by most other commercial suppliers 
of thermoelectric generators and is new to automobile applications. The 
demonstrated LPTG-prototype consists of 20475 thermocouples made of a 
doped bismuth telluride alloy (p-leg: (Big gsTeo 75)2Te3, n-leg: BigCSeQ iTeg 9)3). 
The active films have a thickness of 2.5 micrometer, the internal electrical 
resistance amounts 3.8 kOhm, see figure 2. The volume of the packaged ther- 
moelectric generator is 0.5 cm^. 

The figure of merit for thermoelectric generators, denoted ZT, is given by: 



K 



(1) 
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where S is the Seebeck Coefficient, s the conductivity, T the temperature and 
K the thermal conductivity. The figure of merit is an expression for the effi- 
ciency of converting thermal energy into electricity. The described thermo- 
electric generator has a Seebeck Coefficient of 0,26 V/K and a thermal con- 
ductivity of 0,161 W/K. The focus of design optimization has been maximum 
power density. This has been achieved at the cost of reduced efficiency. 
Commonly used thermoelectric generators are made of doped bismuth tel- 
luride alloys in bulk technology and achieve an efficiency as high as 6%. The 
described thin film devices, however, only achieved an efficiency of about 1 % . 




Fig. 2. Photograph of a thermoelectric generator using foil technology 
(LPTG) and a conventional thermoelectric generator using standard 
hulk technology. 



3 Characterization of the Thermoeiectric Generator in an Engine 
Environment 

The thermoelectric generator was tested under conditions as they are typical 
in the vicinity of the engine. For that purpose an oilpan of a 320 CDl diesel 
engine was modified so that a thermoelectric generator and cooling fins could 
be mounted (figure 3). 




Fig. 3. Thermoelectric generator mounted on a 320 GDI diesel engine of a 
Mercedes-Benz S-Class., 
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Experiments with the oilpan were performed in the laboratory so that a wide 
range of test conditions could be covered. For that purpose the forced cooling 
by the airstream during driving is simulated by a van. Prior to the experiment 
we measured the flow velocity of the air at different locations near the pow- 
ertrain and modeled the local airflow near the cooling fins as function of the 
speed of the car. This model served as input for the airflow supplied by a van 
for forced cooling of the cooling fins in our setup. Using such setup we meas- 
ured an output of the thermoelectric generator of 2 mW (AT = 20 K) and 6 mW 
(AT = 40 K), shown in figure 4. The characteristic curve of the generator out- 
put is plotted in figure 5. The internal resistance measured at a temperature 
difference of 45 K decreases by almost 50% compared to measurements with 
a temperature difference of 1 5 K. 
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Fig. 4. Generator output as function of temperature difference between 
mounting site on oilpan and temperature measured on the cooling 
fins of the cold site. 



230 I Powertrain 



Ku2 DTS-TGB 11 



2500 
2000 

3 1500 

g 1000 

o 

500 
0 

0 1 2 3 4 5 6 7 8 9 10 11 12 13 

output voltage [V] 

Fig. 5. Characteristic curve of generator output as function of tempera- 
ture difference between mounting site on oilpan and temperature 
measured on the cooling fins of the cold site. The internal resist- 
ance measured at a temperature difference of 45 K decreases by 
almost 50% compared to measurements with a temperature dif- 
ference of 15 K. 
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4 Conclusions 

We showed that a single chip of the characterized thermoelectric generator is 
ahle to supply more than 6 mW at a temperature difference of 45 K. The reli- 
ahility of this technology still needs improvement. The focus will be on pack- 
aging of the chip, especially stacking the foils to built a package. Our experi- 
ments furthermore proved the remarkable power density achieved using the 
described technology. However, the high thermal conductance of each foil 
layer enables a considerably thermal flow to take place. This behavior is char- 
acterized by the comparable low figure of merit of 1 % of this particular con- 
struction of a thermoelectric generator. A consequence is that a large heat sink 
is required. The power density per cubic centimeter is thus much lower than 
for a conventional thermoelectric generator. The overall package dimensions 
of sensor systems is crucial for automotive applications. As result, a conven- 
tionally fabricated thermoelectric generator with state-of-the-art high ZT 
materials is suggested for future work. 
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Abstract 

A new European On Board Diagnostic system (EOBD) for real time 
monitoring of emissions from gasoline engines has been developed, 
set up and applied in order to satisfy the Euro 3 and Euro 4 regula- 
tions. The new system has been required in order to control the per- 
formances of fuel injection, closed-loop control and three-way cata- 
lyst [1], 

The EOBD system has been legislated by the Directives 98/69, 
2002/80 and 2003/76 which are mandatory for Euro 3 and Euro 4 
configurations. Therefore, the EOBD diagnosis has been developed to 
check the following systems/components: 

► Catalyst efficincy. 

► Oxygen sensor. 

► Misfiring rate. 

► Fuel system components. 

This article reports preliminary indications to perform a catalyst effi- 
ciency monitoring. A computational procedure is applied for evalu- 
ating the oxygen amount delivered by the catalyst in order to indi- 
rectly estimate the catalyst oxygen storage capacity. The EOBD has 
been applied with the new catalyst layout (pre and main, close cou- 
pled, etc.) in order to meet Euro 3 and Euro 4 Standards (2000/2005). 



1 Emission Behaviour in Durabiiity 

In order to better characterise catalyst durability performance, a new type of 
emission representation is now frequently used; it is the so-called "cumulated 
emissions". Figure 1 shows an example of the three pollutants when driving 
the MVEGB European cycle (ECE-i-EUDC). The diagram outlines catalyst 
behaviour during mileage accumulation in four steps: 0; 15.000; 60.000 and 
80.000 km (50.000 miles). This representation gives the advantage of easily 
representing the importance of ageing linked to cold- and to hot-cycle phases. 
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Fig. 1. Cumulated emissions; Fiat Punto 1,2 Liter Euro 4. 



This set of measures refers to a prototype layout: a pro - and main- catalyst sys- 
tem (0.3 and 1- litre) applied to a 1.2 1; 8 valve engine in order to achieve Euro 
4 level [2]. 



It is possible to distinguish the effect of the light-off temperature increase, cold 
phase, from the factors capable to deteriorate the hot phase: loss of surface 
area and accumulation of metals capable to poison active wash coat sites. 
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The main properties that improve light-off are: positioning of the catalyst clos- 
er to exhaust manifold, reduction of catalyst/exhaust thermal capacity, proper 
noble metal selection and cross-frontal area. Some effects that worsen hot- 
phase performances also impair the cold-phase ones. It is worth mentioning 
that EOBD monitoring is not effective during the warm-up phase, and its diag- 
nostic capability can take place only during the hot phase. 



2 Catalyst Efficiency 



After having reached the light-off temperature, catalyst efficiency is very 
high. Figure 2 shows the emission reduction when A/F control is maintained 
on the stoichiometric value. Oxidation and reduction reactions are well known 
and very effective. Unfortunately A/F control is not always so close to stoi- 
chiometric as desired; on the rich side it is useful to outline the beneficial 
effect of water vapour, which promotes the water shift and the steam reform- 
ing reactions in order to lower CO and HC. On the lean side, NO^ rise is very 
steep [3]. 




Fig. 2. Tailpipe emissions versus A/F. 



Figure 3 shows the effect of ceria reactions capable of absorbing oxygen in the 
lean side and to release it in the rich side. In other words, its presence is capa- 
ble of widening the stoichiometric window. Figure 4 outlines the fact that 
noble metal and ceria are very well distributed [3]. More specifically figure 4a 
shows both ceria and precious metal particles with high surface area, while 
Figure 4b is representative of a catalyst that still presents a sufficiently high 
surface area, but has lost much of its ceria surface and its interaction with the 
metals. 
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(SAE n“ 931034) 


Water gas diift 
C0 + H:0 = C 03 + H2 




Steain reforuiiiig 

CxHy + ZxHjO = XCO 2 + [2x + (y/2)] H: 




“Oxygen Storage” reactions 


Cei'ia released 

CO + 2CeOj = CO 2 + CejOj 


C«’ia stored 

O 2 + 2CeiOj = ‘ICeO: 


RICH SIDE 

◄ 


LEAN SIDE 

► 



A/F 

Fig. 3. Additional reactions. 

Therefore, when the catalyst reaches its proper operating temperature, the fol- 
lowing key parameters are essential for obtaining the expected performances: 

► Narrow A/F control both in steady state and in transient conditions. 

► Proper Geometric Surface Area (GSA). 

► Noble metal quantity and its distribution on the wash coat. 

► Relative distribution between noble metal and ceria. 

► Relative ceria/noble metal quantity. 

► Proper Oxygen Storage Capacity (OSC). 



3 On Board Diagnostic to Monitor Cataiyst Efficiency 

Catalyst efficiency monitoring is performed by measuring the OSC, which is 
the only parameter that can be reasonably and continuously monitored on the 
car, installing a second oxygen sensor downstream of the catalyst. By compar- 
ing upstream/downstream oxygen sensor signals, it is possible to obtain an 
indication of the residual OSC. Dir. 98/69 states that the malfunction indicator 
must be switched on when HC level, measured on the complete MVEGB cycle, 
reaches 0.4 g/km. This level is known as the "Threshold" and can be compared 
with the Type Approval Standard (TAS) equal to 0.2 g/km applicable to Euro 3, 
which must be fulfilled for 80.000 km. For the time being the Threshold value 
remains unchanged, even if Euro 4 HC Std. has been reduced to 0.1 g/km for 
100.000 km. 
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Fig. 4. Active site distribution with fresh and aged catalyst. 

OBD catalyst monitoring is performed measuring only HC emissions, it is 
therefore clear that the upstream and downstream oxygen sensors must be 
capable of distinguishing between a normal OSC loss of up to 0.2 or to 
0.1 g/km, and an unusual loss responsible for emission degradation up to 
0.4 g/km. Defining Euro 3/4 catalyst is important to guarantee both the sta- 
bility of the traditional parameters (A/F control, GSA, etc.) as well as the sta- 
bility of OSC. In this respect the zirconia - ceria mixed oxides have provided 
the best temperature-stable oxygen storage to date, and are used extensively 
in close-coupled applications. In fact converter temperatures of 1.000°C are 
above the phase transition temperature from gamma alumina to alpha alumi- 
na. Stabilized alumina is the material that provides the largest surface area on 
which the active catalytic ingredients have been dispersed for stable opera- 
tion. When the pores of the catalyst support collapse in the phase transition, 
they bury the active ingredients and the oxygen storage rare earth materials 
[4]. 

Mixed oxides of ceria and zirconium have provided a suitable support surface 
for the close-coupled catalyst with excellent thermal stability. These tempera- 
ture-stable oxygen storage components have also provided the basis for the 
design of oxygen monitors that activate the malfunction indicator light. 
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4 Catalyst Deterioration 

It is wise to underline, figure 5, how the simple heating of the catalyst sub- 
strate in an oven with stable air at 1.300°C for 64 hours is less severe than 80 
hours at 1.100°C, but with substantial A/F ratio variations ( + 25%). When dis- 
cussing artificial catalyst ageing for failure simulation, this a key issue if the 
temperature is to be raised in stable air, or the catalyst is exposed to lower tem- 
peratures while determining A/F ratio variations. According to our experi- 
ences, this latest procedure appears recommendable and the first one is loos- 
ing its importance. 




Fig. 5. Cumulated tailpipe emissions measured with catalyst having dif- 
ferent aging. 



5 Oxygen Sensor Responses and OSC Determination 

The quantity of oxygen stored in a typical catalyst volume (1. 5-2.0 litres) 
ranges between 1.100 and 1.500 mg. When the catalyst is aged, but still sat- 
isfies 80.000 km Euro 3 legislated value, HC = 0.2 g/km, OSC ranges between 
100 and 300 mg. This value drops below 50 mg when OSC has been seriously 
reduced and HC level approaches the 0.4 g/km value, therefore determining 
the switch-on of the malfunction indicator. Figure 6 shows the correlation 
between Oxygen Storage Capacity and tailpipe emissions. 
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Fig. 6. Oxygen storage capacity variation versus tailpipe HC measured 
driving MVEGB cycle. (SPM: engine test bench at 1.000°C). 



Evaluation of oxygen quantity is carried out by measuring the delay time of 
downstream oxygen sensor when a sudden A/F ratio variation is induced 
upstream. 

Figure 7 shows an A/F step variation, from lean to reach, measured by the two 
upstream lambda sensors HEGO (on/off type) and UEGO (linear type); after 
approximately 10 s also downstream signal is modified accordingly. In the 
upper part of the same figure is shown the behaviour of two UEGO sensors 
used as control. It is also outlined the time window determined during the two 
transient conditions with the signal B1 - B2. A similar behaviour is recorded 
switching from rich to lean. Upper part of the figure shows the behaviour of a 
new catalyst, that records a difference, between the two HEGO signals, close 
to 2.5 s. 
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Fig. 7. OSC for new and aged catalyst (B1 - B2), HEGO and UEGO signals. 

The lower part of the same figure reports the behaviour of an aged catalyst; 
during the A/F transient it is evident that the HEGO signals are the same con- 
sidering that OSC is close to zero. 

Response delays of downstream HEGO compared with the one upstream are 
very well linked with oxygen storage quantity still available on catalyst sur- 
face. It is possible to determinate OSC during A/F variations using formulae 
specified in (1) and (2): 



Oxygen released (mg O 2 ) 






I 



1000^ lOOOni^ 



3600 ■ \kg 



1 ;? 



.vO.23 ]dt (1) 
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Oxygen stored (mg O 2 ) 



OSC. 



rich 10 Icanh 






1 1000^ lOOOffig 

3600 \kg Ij 



At).23 ]clt (2) 



Where: 

► Oxygen content in ambient air reaches 23%. 

► NTK probe reference signal upstream catalys. 

► to rich: time in seconds of transient window (the first transient rec- 
tangular window). 

► to lean: time in seconds of transient window (the second transient 
rectangular window). 

Starting from the above it is possible to define simplified correlation: 

O 2 mass released = 0.23x Air Flow/3600 x (!->.) x {Ty^j„^„stream ' 'I'\up^treaJ 
for < 1 

O 2 mass stored = 0.23x Air Flow/3600 x (>^-l) x {Ty^^dowmtream ' '^"k^upstrean^ 
for 1 > 1 



Where: 

T\upstream ^^d downstream I®] a^e the time span to reach 90% of the final value 
during the rise of signal induced by mixture enrichment and the time span to 
reach the 95% of the final value during the signal decrease induced by the 
mixture leaning 

Air - flow [kg/h] is the air induced through engine intake 
A numerical example of 02-stored determination is reported [5]: 

0.23 X 50/3600 x (1. 1-1.0) x 2.5 = 800 mg 

Where: 

0.23 is O 2 content in ambient air [kg/h] 

(1.1 - 1.0) lambda variation 

downstream ' '^k,upstrean^ “ 2.5 S 



6 Measurable Parameters on Car 

Figure 8 and 9 [5] show a typical response of downstream HEGO (VLAM2) for 
a new catalyst and for a one definitely aged in comparison with the upstream 
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HEGO signal (VLAM 1). In particular, the numerical catalyst efficiency is 99% 
for the new catalyst and 16,94% for the aged one. It is clear that VLAM2 fre- 
quency is very low in the first case and high in the second case; in this later 
case its frequency becomes similar to the one of upstream sensor (VLAMl). 

Starting with cold engine, during the first 300 s of the MVEGB cycle, the 
VLAM2 signal is not active because it does not reach its operating condition. 

The useful signal comparison takes place only when the engine reaches its 
thermal operating condition and the diagnosis becomes active. 

At this moment the ECU activates DCASTABT function, which systematically 
is active to check when proper conditions are reached (catalyst temperature, 
engine rpm, vehicle speed, etc.) and to calculate catalyst efficiency. 




Fig. 8. Determination of new catalyst efficiency. 

In figure 8 catalyst efficiency reaches 99,99%, in figure 9 this value drops to 
16,94%. 
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Fig. 9. Determination of aged catalyst efficiency. 

The emission level that activates the malfunction indicator lamp (MIL) corre- 
sponds to 0.4 g/km of HC during the MVEGB cycle. This condition takes place 
when the index of catalyst efficiency, DCAINDST, is about 60%. 



7 Conclusions 

The new on-board diagnostic system for the analysis of pollutant emissions 
from gasoline engines has been developed, set up and applied in order to sat- 
isfy the Euro 3 and 4 regulations. In the built-up EOBD system - European on 
Board Diagnostic system - four distinct diagnoses have been developed for 
detection of faults that are capable of inducing pollutant emission levels, for 
each kilometre covered by car, exceeding the regulation limit. In this paper the 
Catalyst Diagnosis has been described and discussed. A computational proce- 
dure has been developed for evaluating the oxygen amount stored by the cat- 
alyst in order to estimate indirectly the Catalyst Oxygen Storage Capacity. This 
parameter is the only one capable to detect catalyst efficiency during normal 
engine operations after initial start-up and thermal transient. 
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Abstract 

The SIRPA project aims at developing a multifunction automotive 
sensor with the integration, on a single CMOS array, of several sens- 
ing functions presently provided by different sensors. Such multi- 
functional sensor will be used to detect a number of critical envi- 
ronmental parameters, such as the luminance of the background and 
the visibility (in the presence of fog, rain and windshield dimming), 
while providing additional information about the driving scenario 
(e.g. curvature of the road, oncoming vehicles, approaching tun- 
nels, ...). The project was launched by Centro Ricerche Fiat and 
involves Neuricam, ITC-IRST and University of IFento. The sensor is 
meant to provide some of the required input data for the automatic 
control of the adaptive headlamp system developed by CRF. In this 
system the headlamp beams can be modified according to the par- 
ticular driving and environmental conditions. The prospect of inte- 
grating several functions into a single optical CMOS required 
redesigning existing functions and developing new ones (fog detec- 
tion). The approach here used was to develop and validate the indi- 
vidual functions separately and afterwards to integrate them onto a 
CMOS array. 



1 Introduction 

Nowadays a lot of sensors are used on vehicle or are being developed to detect 
environmental parameters (luminance, rain, dimming, ...) and the driving sce- 
nario (oncoming vehicles, approaching tunnels, lane detection, ...). These sen- 
sors provide the necessary input for comfort and safety assistance systems: 
illumination adapting to the visibility conditions, fog lamp lighting, turning on 
(off) lamps before tunnel entrance (exit), illumination adapting when crossing 
other vehicles, automatic activation of windscreen wipers, automatic demist- 
ing, automatic speed control, lane warning, etc. 

Car makers have to manage this increasing number of sensors which means 
complexity in terms of sensor housing, cabling, electronic interfaces and actu- 
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ation strategies. For this reason, component suppliers are working toward the 
integration of more functions. Yet, at the moment the integration level is lim- 
ited to a maximum of three different sensors in a single package and the func- 
tions involved are non-imaging. 

The monitoring of the area in front of the vehicle with one or more cameras 
allows to collect useful information on the scenario (kind, dimensions and 
shape of objects and obstacles) to implement comfort and safety functions. 
Lane warning and lane keeping systems, which are still in progress, are based 
on the use of a camera (CCD or CMOS array) placed close to the internal rear- 
view mirror. The format of the camera is often redundant for the function (e.g. 
VGA area is not completely used in lane warning function). 

The former needs (combination of imaging and non-imaging functions, and 
reduction of number of sensors) can be successfully addressed by a multifunc- 
tional integration on CMOS array. The advantages stemming from this solution 
are: increased reliability, easy-to-use functions control, reduction in number of 
sensors and components, lower size, easier mounting on vehicle and reduced 
total costs. 



2 Adaptive Headlamp System 
2.1 Description of the System 

The Adaptive Front-lighting System (AFS) is a frontal illumination system 
which can "adapt" the lighting beam configuration to the particular driving 
condition. Within EURIMUS EM34 project [1], Centro Ricerche Fiat developed 
an adaptive headlamp system characterised by the working modes sum- 
marised in the following table. 
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1 Mode 


Requirements 


Features | 


Town fightmg (TL) 


1 0 lUiumnate pedestnan 
To avoid dazzliuji 


Wide latei'al illumination 


Country lighting (CL) 


To illujiiinate roadside markers 
To a\'oid dazzling 


Smiittt' to the standard low* 
beam, but it itliuumnlca iartlicr 
md ba.s awider angle 


Motorway lighting (MI^ 


High speed driving 

Motorway lanes 

To avoid dazzling from rear- 


Long I'itfige tihmiinarion 


Rending lighting (RL) 


view miiTor 

To illmmnate tlie curve in 


Beam is pointed left/right 


advance 






Wider lateral iUmnination, 


Adx'erse conditions lighting 


To avoid dazzling from wet 


less frontal illiuiiiiuiliou 


(AI.) 


pyoxmd 


(that generates glare) 



Table 1 . AFS working modes. 



Each mode has its peculiar light distribution (range and divergence) fulfilling 
the requirements of the correspondent driving scenario. The desired beam 
configuration is obtained through an optical system, based on micro optics 
besides the traditional optical elements. The optics lateral movement is based 
on piezo-actuation, which provides to change the beam configuration in a suit- 
able time interval. In figure 1 the Lancia Kappa vehicle demonstrator and some 
examples of beam configuration are shown. 




Fig. 1. Lancia Kappa, demonstrator vehicle equipped with adaptive head- 
lamp system. 

In order to switch automatically from one mode to the other, the adaptive 
headlamp needs several sensors to recognise the driving condition (vehicle 
speed, steering wheel angle, ...) and the driving scenario (street geometry. 
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presence of curve, oncoming vehicles, ...), and to measure the environmental 
parameters (illumination, weather conditions, ...)• 



2.2 Functional Requirements of the Sensors 

This paragraph describes the functional requirements of the main sensors pro- 
viding data on the driving scenario and the environmental parameters. 

The fog function measures the density of the external fog to turn the adaptive 
headlamp beam to the AL mode. The output is the visibility level in metres, 
within a range of 5 m-150 m, with ±20% confidence. In particularly critical 
visibility conditions, the system can decide to disable the output of the other 
optical sensors which could provide wrong data (e.g. curve monitoring). 

The rain function is aimed at detecting the presence of rain droplets (or snow) 
on the windshield, in order to actuate the windshield wipers and change their 
speed according to the rain intensity, and to turn the lighting beam to the AL 
mode, which minimizes reflections from the wet ground. The sensor should 
discriminate 5 levels of rain intensity and also phenomena like splash or dew 
on the external part of the windshield. The device has to work properly also in 
non-ideal cleaning conditions, due to the wear of the windshield wipers. 

The twilight function measures the ambient illumination, in order to turn the 
vehicle lamps on and off automatically in case of scarce or good illumination 
conditions respectively. Beyond the 'twilight threshold' (approx. 5-10 lux), 
which is the most important requirement for the adaptive headlamp system, 
the discrimination of more visibility levels (in the range 1-10® lux) can be used 
for controlling the vehicle climatic system. 

Similarly, the tunnel function detects the presence of a tunnel or a very dark 
passage, in order to automatically switch on and off the car lamps when enter- 
ing and leaving a tunnel. In particular, the entrance of the tunnel has to be 
detected in advance, taking the actuation time in account. 

Crossing vehicle detection allows to optimise the use of TL and ML modes. 
Vehicles travelling either in the same or in the opposite direction with respect 
to the driver are detected, with such an advance as to allow the switching of 
the beam modes before dazzling the crossing or preceding vehicles. 

Finally, the scene in front of the vehicle has to be monitored, in order to detect 
the presence of a curve in advance with respect to the data provided by the 
steering wheel sensor, and to allow the system to turn to the CL mode. 
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Noise such as ambient light, flashes, dirt or dimming on the windshield, can 
influence the function performances. Several working conditions should be 
therefore taken into account, in order to achieve the required robustness and 
reliability. 

The internal dimming of the windshield has also to be monitored (possibly in 
a predictive way, i.e., with more sensitivity than the human eye) and serves as 
a warning for the automatic activation and regulation of the air climatic sys- 
tem. This function does not provide direct input to adaptive headlamp system, 
but it is necessary to enable the above mentioned detection functions. 



3 State-of-the-Art 

The most automotive environmental sensors integrate more sensing functions 
(maximum of three different sensors) at package level: 

1. rain and light sensors (e.g. by Bosch, Hella, Kostal and Valeo) perform- 
ing rain, tunnel and twilight detection; 

2. illumination sensor (e.g. by First Technology), which can detect the pas- 
sage through tunnels, the twilight condition and the sun irradiation 
inside the vehicle. 

The optical detectors are mostly photodiodes, so that no imaging function is 
actually performed. The measured quantity is the collected radiation intensi- 
ty, which is typically: diffused light from the outside scene for twilight and 
tunnel functions; direct sunlight for solar functions; scattered/reflected NIR 
radiation in case of rain sensing. The combination of signals coming from dif- 
ferent sensors is then analysed through suited algorithms. 



4 Multifunctional Sensor [2] 

4.1 Sensor Design Based on CMOS Array Partition 

The SIRPA [3] project aims at developing a multifunction automotive sensor 
with the integration on a single CMOS array of part or all the above mentioned 
functions, presently performed by different sensors. The approach, claimed by 
a patent, is to utilise the array as the photo-sensitive element and to divide it 
into sub-areas, each one being dedicated to one or more functions. The solution 
proposed by CRF will allow to go far beyond in terms of both number and type 
of integrated functions, thus reducing system costs significantly. 
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The acquisition and processing of each area can be optimised by the use of 
windowing [4] (i.e. reading of sub-areas of the pixel array) and the use of the 
suitable frame rate and sensor parameters (e.g. gain). 

First of all, the concept design was performed, in order to choose the most 
advantageous sensing 'principle' for each function. The main characteristics of 
the single functions are reported in the following table. 



Function 


Actiife passive* 


Detection principle 


Optical Fttieri/tg 


Orientation 


Area 

Htonitormg 


PassK'e 


Image analysis 


Polarizer 


Horizontal 


Rain 


Active 


Image analysis 


Band'pass 


Oithogonal to windshield 


Mistmg 


Active 


Image analysia'intensity level 


Band-pass 


lii different 


Fog 


Active 


Intensity levels 


Band-pass 


Honzoulai or up\vai*d 


Twitigbt 


Passive 


Intensity lev'els 


None 


Upwaixl 


Crossing 

vekiclt 


PassK’e 


Image aualysi$''iuteiisity le\’el 


Polarizer 


Honzoutal 


(*) Actire- 


wWt emitter; passive- rviStoui emitter 







Table 2. Main characteristics of simple functions. 



From the functional specifications and concept design, the resolution and field 
of view (f.o.v.) were evaluated. These parameters yielded the focal length; 
thus, the only degree of freedom for the /^number was the lens section. This 
was not a very strict constraint for active functions- because a narrow aperture 
of the lens could be partially balanced by increasing the emitter power (with- 
in ECE safety norms) - while it represented a critical issue for passive imaging 
functions, such as the area monitoring. Yet, there was a further specification 
in our design which actually revealed itself advantageous to solve the problem 
of integration: it was the viewing orientation. For some functions, like area 
monitoring, the sensor must "see" in front of the vehicle, while for others (e.g 
rain) it must be pointed orthogonally to the windshield to have uniform reso- 
lution in the image plane. On one side, that set a technical problem of optical 
axis deflexion, while on the other side it gave a further degree of freedom to 
the optics sizing. 

Another important factor determining the system geometry was the isolation 
of the optical sub-systems. Indeed, the image planes on the CMOS array are 
very close to each other (approx. 100 pm separation), so we had to face the 
problem of noise due to stray light. 

The following figure shows an example of CRF proprietary array partition [5,6] 
in which each sub-area includes both image plane and blind pixels for separa- 
tion; in particular, it corresponds to the first prototype, which is described in 
chapter 4.2. 
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Fig. 2. VGA CMOS array partition. 



4.2 Optical Design of First Prototype [7, 8, 9] 

A first prototype of multifunctional sensor was developed (figure 3), integrat- 
ing onto a single VGA CMOS array the following functions: visibility, twi- 
light, tunnel and frontal area monitoring. 



(d) 



(a) 

Fig. 3. First prototype: optical holder to be mounted on VGA CMOS array 
(45 X 35 X 15 mm); (a) frontal monitoring optics; (b) NIR-LED for vis- 
ibility function; (c) receiver lens for visibility function; (d) top of the 
optical fibre for twilight function. 
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For the frontal monitoring optics, it was decided to use wide angle objectives 
(/’ = 6-8 mm) of small size (10 mm diameter by 15 mm length). With the opti- 
mal configuration view (f = 7.5 mm) the objective and CMOS array are ori- 
ented about 5° from the horizon to the ground, and, with the f.o.v. and area 
specified in figure 2, allow for the monitoring of a 3.5 m wide lane of the road 
from 5 m on. 

For the tunnel function, a dedicated sub-area was chosen inside the frontal 
monitoring one, having f.o.v. of 10° horizontally and vertically, and pointing 
towards the horizon. As described in chapter 2.2, this configuration allows for 
the recognition of a tunnel at about 40 m distance. Finally, a polarizing filter 
was placed in front of the optical system, in order to minimize reflections from 
the vehicle dashboard. 

For the visibility function, the backscattering technique [10, 11] (emission of 
NIR beam and collection of the radiation scattered by fog particles) seemed the 
most promising, because the emitter can be placed close to the detection sys- 
tem, thus maintaining the overall size small. An emitter (NIR-LED) and the 
receiver collecting lens have respectively the beam divergence and the field of 
view with partial overlapping. In presence of fog the concentration of small 
water particles in the overlapping zone produces a backscattering towards the 
receiver, with an intensity which is proportional to the fog density. Since there 
are no visibility sensors in the automotive field, a preliminary prototype of fog 
sensor with a PIN photodiode was developed with good results. Its small size 
(20 X 20 X 30 mm2) makes it suitable to be integrated within the internal rear- 
view mirror. For the CMOS based sensor, the collecting lens size was reduced, 
the emitter power enhanced and the field of view optimised. An optical filter 
was added for S/N ratio enhancement and a plastic optical fibre (1 mm diam.) 
was placed after the receiver lens. With the optical fibre carrying the collect- 
ed signal, the receiver lens (8 mm diameter) could then be placed far enough 
from the CMOS as to give space to frontal monitoring optics. 

Another optical fibre was used for the twilight function. The light is collected 
in a cone of view having 30° semi-aperture and pointing 60° upward. Both fog 
and twilight fibres are held close to the CMOS dye (400 pm distance), in order 
to optimise signal intensity (the fibre output beam is indeed divergent). 

The problem of optical isolation of the different areas was solved by using an 
optical window, covered by an absorbing serigraphic coating (mask) on the 
side facing the CMOS. Such mask, being only 400 pm far from the dye, limits 
the image planes of the optical subsystems and reduces stray light to tolerable 
values. In the design of the mask, the manufacturing tolerances of the CMOS 
package had to be taken into account. For this first prototype, a fine-position- 
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ing sYstem was realised to balance the mechanical tolerances which might 
affect the optics alignment. 



4.3 Hardware Architecture [12, 13] 

Figure 4 shows the hardware architecture of the adaptive headlamp system 
which is made of a digital image capturing block (Camera "Eye"), image pro- 
cessing and control unit. 

CONTROL UNIT 



Micro 

Controller 



J L ADAPTIVE 
HEADLAMP 

lActuatore] |Lights| 



Fig. 4. System architecture of the adaptive headlamp system. 

Camera "Eye" is a digital image capturing system specifically designed for 
automotive and outdoor applications. It is mainly composed of a 0.35 pm 
CMOS array, an FPGA controller and some peripheral ICs. The sensor, supplied 
by Neuricam, is a monolithic monochromatic VGA active-pixel which inte- 
grates: pixel array, video amplifiers, 10-bit ADC, microprocessor interface and 
several other support blocks. The response of the pixel is a logarithmic func- 
tion of the light intensity which allows to cover a high-dynamic-range 
(120 dB). The FPGA controls the transfer of data from and to the sensor; it is 
programmed to acquire blocks of pixel (ROls) and to perform fixed-pattern- 
noise correction at frame rate. The frame rate is 24 full frames per second. 

The image processing unit, which was developed by CRF Engine and Vehicle 
Division for Lane Warning, is used to acquire and elaborate the digital video 
stream coming from Eye module. It is composed of a DSP (150 MHz, 24 bit) for 



CAMERA "EYE" 



IMABE PROCESSING UNIT 
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the real-time image processing and a microcontroller (50 MHz, 16 bit) to con- 
trol the I/O signals (digital I/O, CAN B - C, RS232) and the overall system set- 
tings (l^C bus). 

Finally, the electronic control unit based on micro controller implements the 
adaptive headlamp strategies by different outputs (6 actuator drivers, 8 rele, 
4 PWM, CAN B - C and RS232). In figure 5 the overall system hardware is 
shown. 




Fig. 5. System hardware: Eye and image processing unit (left), electronic 
control unit (right). 



5 Results 

5.1 Fog Detection 

As anticipated in the former paragraphs, a first prototype of visibility sensor 
for automotive applications, based on a PIN photodiode, has been developed 
(fig. 6). Visibility range of about 150 m, reduced size, high reliability and low- 
cost ( < 1 5 Euros) are the main features. It is composed of the following com- 
ponents: 

► Two pulsed NIR LEDs, with a cylindrical lens to shape the emitted 
beams, 

► PIN receiver with lens to collect backscattering radiation and an elec- 
tronic filtering stage, 

► PIN to verify the cleanness of the windshield in front of the fog sensor, 

► PIN to check the LED power. 

The validation measurements were carried out at the "Laboratoire Regional 
des Ponts et Chaussees" in Clermont-Ferrand (F), which has a tunnel for artifi- 
cial and controlled fog generation. 
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• LED wavelength 

• LED beam divergence 

• Measurement distance 

• Visibility range 

• Analog output 

• Size 

• Power supply 



850 ^ 880 nm 
5 X 20 ’ 

5 30 cm 
10 -150 m 
0-8 Volt 
21x 33 X 24 mm 
12 V 



Fig. 6. Automotive fog sensor prototype and specifications. 



In figure 7 the visibility level (m), measured by a transmissometer, is plotted 
versus the analogue output (V). 
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Fig. 7. Visibility vs. sensor analogue output. 

Also for the CMOS array based prototype a set of validation tests have been 
done in fog-chamber. In figure 8 the visibility level within a range of 5-500 m 
is plotted versus the average grey scale level (ROI of 55 x 55 pixels). 




258 I Comfort and HMI 



600 




520 525 530 535 540 545 550 555 560 565 570 

Average grey scale level [a.u.] 

Fig. 8. Visibility vs average grey scale level. 

The intensity of backscattering radiation scales approximately as the inverse 
of visibility, therefore the output signal is nearly linear, due to the logarithmic 
response of the CMOS pixels. 



5.2 Twilight Detection 

Figure 9 shows the transfer function between the external illuminance, meas- 
ured by a luxmeter, and the average grey scale level (ROl of 55 x 55 pixels). 
The CMOS logarithmic response allows a good resolution over a wide range of 
lighting conditions without the need of iris adjustment. In order to discrimi- 
nate different environmental scenarios, specific ranges must be chosen: twi- 
light (0-10 lux), overcast sky (10-1000 lux) and indirect solar light 
(>1000 lux). 
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Fig. 9. Illuminance vs average grey scale level. 



5.3 Tunnel Recognition 

The aim of this function is to recognize the approach and the passage through 
a tunnel and to discriminate it from a bridge. The simple and fast algorithm is 
based on the comparison between two different ROls contained within the 
tunnel function subarea, as shown in figure 2. Several tests were carried out 
with the vehicle demonstrator in order to develop the algorithms and to 
improve the function robustness. In particular, different roads (urban and 
extra-urban, motorways) and light conditions were considered. 

In figure 10 the algorithm response versus the acquired frame during a typi- 
cal approach and transition into a tunnel is shown and compared with the out- 
put of a commercial twilight/tunnel sensor. The developed function is able to 
recognize in advance the tunnel entry (and exit) and this is very important 
when the vehicle speed is high on motorway. 
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Fig. 10. Example of tunnel analysis. 
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In the test site above, the car runs at 83 km/h and the algorithm recognises 
the tunnel entry and exit respectively 35 m and 46 m before. 

Another important characteristic is the capability of discriminate the tunnel 
from a bridge. In the next figure the response of our sensor is compared with 
a commercial tunnel/twilight sensor. There is not the negative peak, charac- 
teristic of tunnel recognition, and the positive peek, associated to the exit, is 
lower than the tunnel exit peak. 
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Test site: Beinasco bridge (Torino-iTALY) 




Frant« 



Fig. 11. Example of bridge analysis. 



5.4 Curve Monitoring 

The developed function belongs to the class of segmentation problems, whose 
main goal is to divide an image into parts having a strong correlation with 
objects or areas contained in the image. Therefore, the core of the algorithm is 
based on methods of thresholding, edge and region segmentation. In the spe- 
cific class of edge-based image segmentation, the Hough transform segmenta- 
tion was used to detect objects of known shape (lane) within the image. 

In figure 12 an example of road scene with the detected lanes are shown.The 
angle between lane and vehicle directions and the distance of vehicle from 
each lane (left and right) are evaluated from the acquired and processed 
images. Then, the outputs are sent to the control unit of the adaptive head- 
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lamp system to bend the lighting beams in advance with respect to the infor- 
mation provided by the steering wheel sensor. 




Fig. 12. Example of lane and curve detection. 



6 Developments in Progress 
6.1 Optical Solutions 

Presently, a novel optical solution is being studied, considering non-standard 
optical elements, such as micro-lenses arrays. The goal is to increase the inte- 
gration level by replacing discrete optical components with structures consti- 
tuting of more components (lens, prisms) into the same plastic element. A first 
solution has already been designed: it is a hybrid system, composed of both 
lens arrays and traditional lenses, having the suited characteristics to imple- 
ment the former functions (visibility, twilight, tunnel and curve monitoring) 
and to add new ones (rain, misting and crossing vehicles). Actually, the algo- 
rithms performing the rain function are still in progress, as explained in the 
next paragraph. 



6.2 New Functions to be Integrated 

The task of recognizing rain and misting conditions can be attained through 
the illumination of a limited area of the vehicle windscreen by an NIR source. 
Without rain drops on the windshield, no NIR light of the emitters reaches the 
sensor. On the contrary rain drops act as curved mirrors, reflecting the source 
image in direction of the collecting optics and are thus revealed by edge-detec- 
tion algorithms. In case of misting, a Lambertian scattering takes place and the 
source appears like a halo, with different intensity - more or less bright - and 
position in case of internal or external misting [14, 15]. 
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Another function which is being developed is the crossing vehicle detection. 
The main requirement is to detect the vehicle with a considerable advance, in 
order to allow the adaptive headlamp system to change lighting mode before 
the dazzling happens. The fact that we are dealing with the recognition of car 
lamps is advantageous, because the light spot appears larger than the size of 
the illumination optics (which is approximately 15x15 mm^ for headlamps). 
The dimension of the required area are approximately 320 x 240 pixels. 



7 Conclusions 

A multifunction automotive sensor with the integration on a single CMOS 
array of several sensing functions, presently provided by different sensor, was 
developed within SIRPA project. It is finalised to detect several environmental 
parameters and to provide additional information on the driving scenario. 
These output are needed for the automatic control of the adaptive headlamp 
system, developed in CRF, in which the lighting beams can be modified con- 
sequently. 

The prospect of integrating several functions into a single optical CMOS 
required redesigning existing functions and developing new ones (as is the 
case for fog detection). The approach was to develop and validate the individ- 
ual functions separately and to integrate them onto a CMOS array. 

The concept of multifunction sensor in terms of CMOS array partition and opti- 
cal solutions has been described. A new automotive fog sensor based on low- 
cost components (PIN and NIR-LED) and stand-alone electronics was developed 
to validate the working principle. Then visibility was integrated onto a multi- 
function sensor together with the curve monitoring and twilight/tunnel func- 
tions. 

The system architecture is composed of the following modules: VGA logarith- 
mic CMOS array, FPGA based camera interface, embedded image acquisition 
and processing unit based on DSP and micro-controller, a control unit with the 
actuation strategies of the adaptive headlamp system. 

The main results of each sensing function have been shown and discussed.The 
project is in progress and further improvements are expected: novel optical 
solutions, considering non-standard optical elements, such as micro-lenses 
arrays to increase the integration level and reduce manufacturing costs; imple- 
mentation of three other functions such as rain, dimming and crossing vehi- 
cles detection. 
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Abstract 

To improve driving safety it is of outmost importance to guarantee 
that the windshield never starts fogging by means of an automatic 
and reliable procedure. To reach this goal and keeping always in 
mind comfort and fuel consumption, a fogging condition estimation 
algorithm has been developed. It requires a Mean Radiant 
Temperature (MRT) sensor and a humidity sensor. The MRT sensor 
is placed on the cabin compartment roof and points towards the 
dashboard and the windshield. Therefore it may be used to estimate 
windshield inner surface temperature. Combining this last piece of 
information with cabin relative humidity measured by the humidity 
sensor, the estimation algorithm forecasts fogging conditions so that 
the automatic thermal control system is able to take actions before 
fogging starts to appear. Centro Ricerche Fiat has already developed 
for Fiat Auto and patented a cabin thermal control that uses as feed- 
back just MRT and blown air temperature one. This approach is not 
only safer, but it allows also cost savings with respect to state of the 
art sensor layout. 



1 Introduction 

One of the most important tasks that the air conditioning system has to per- 
form on a vehicle is to assure perfect visibility by removing moisture con- 
densed on the inner windshield surface. This is obtained by blowing warm air 
with an appropriate velocity and distribution through the defrost outlet. 
Usually, even if there is an automatic climate control unit, the driver is 
required to interact with it to start defog or defrost procedure by pressing a 
button and/or turning on the compressor. 
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This has two consequences on safety: 

► The action takes place when moisture formation is already affecting 
driver visibility. 

► The driver is required to perform an action that is not related to driv- 
ing, thus affecting his/her level of attention. 

Moreover, this may also have an impact on thermal comfort, since 
defrost/defog procedure usually sets airflow to maximum level and tempera- 
ture blending to maximum heat. It may not be possible to assure both befog- 
ging procedure and thermal comfort, since the former is obviously a priority. 
But there are several other strategies that may benefit from reliable estima- 
tion of incipient fogging conditions. In particular, those that require control of 
the recirculation flap, such as air quality systems based on gas sensor, or ener- 
gy saving strategies aimed at reducing thermal load on heater and evaporator. 

Impact on fuel consumption is also worth to be mentioned. One of the devices 
that can help in befogging is turning on the compressor, provided that the 
external temperature is not too low (usually in the climate control unit there is 
a constraint regarding it). This results in condensation of the water vapour 
contained in the air to be treated by the heater, allowing to send drier air to 
the windshield, but it increases also fuel consumption: therefore, to know in 
advance that fogging condition is bound to appear may avoid the need for the 
compressor to be turned on. 

From the above considerations, it is clear that to know when incipient fogging 
occurs can be very helpful for several reasons: safety, comfort and fuel con- 
sumption. 

Generally speaking, there are two ways to get this piece of information: to 
detect it by means of sensors that are directly influenced by the condensation 
itself, or to estimate it by means of models or via sensors that measure physi- 
cal quantities that influence the phenomenon. The different approaches are 
compared in [1]. If the approach requires additional sensors, in the analysis of 
the solution it is very important to evaluate if they can be useful also for other 
functions of the air conditioning system, or if the offer potential for additional 
functions (as an example, humidity control), so to perform the correct cost ben- 
efit evaluation. 

Centro Ricerche FIAT has chosen a sensor set up that include a Mean Radiant 
Temperature sensor and a humidity sensor. This solution allows for incipient 
fogging condition detection and is perfectly integrated with other functions of 
the air conditioning system: thermal comfort and humidity control. 
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2 Technical Background and Sensor Layout Description 

Since 1998 Centro Ricerche FIAT has been using Mean Radiant Temperature 
(MRT) sensors for a better estimation of comfort conditions in car cabin com- 
partments. MRT performs an integral measurement of radiant temperature of 
all the surfaces over a determined field of view. A methodology for identifying 
perceived thermal comfort estimators by means of MRT has been developed 
for Fiat Auto and successfully applied in several prototypes [2]. The approach 
(patented by Centro Ricerche FIAT) requires the MRT to be placed in the mid- 
dle of the roof, pointing towards the windshield and the dashboard, as shown 
in figure 1 . 




Fig. 1. Position and field of view of MRT. 




\ 




It has been shown that this approach allows both sensors number reduction 
(no need for car cabin compartment air temperature sensor and for sun radia- 
tion sensor) and improvement in perceived comfort estimation. Since MRT 
points towards the windshield Centro Ricerche FIAT has started a research 
activity so to evaluate the possibility of using information coming from MRT 
(in combination with a humidity sensor) in order to estimate incipient fogging 
conditions. 

Therefore, since a humidity measurement is necessary, a capacitive humidity 
sensor has been added near the MRT one (see figure 2): 
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Fig. 2. MRT sensor and humidity sensor placed in the car. 



Summing up, the A/C control unit will use the signals coming from the fol- 
lowing sensors: 

► Mean Radiant Temperature. 

► Relative Humidity. 

► External Air Temperature. 

► Blown Air Temperature. 

An algorithm has then heen developed so to be able to forecast incipient wind- 
shield fogging conditions. The block diagram of the proposed approach is the 
following: 
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Fig. 3. Algorithm to be implemented in ACU. 



3 Algorithm Description 

From a theoretical point of view there is the need to estimate or to measure 
dew point temperature of cabin compartment air and windshield inner surface 
temperature, and then to make a comparison between them. To measure 
windshield inner surface temperature would require a dedicated sensor, that 
could not be used for other purposes. Centro Ricerche FIAT has therefore cho- 
sen to estimate it b-y means of a simple, lumped parameters model: 

= G,(r„., - r„ ) + G,(r. - r„ ) (D 

at 

The quantities indicated in (1) are defined here below: 

Windshield inner surface temperature [°C] 

Inside air exchange temperature [°C] 

Ambient air temperature [°C] 

C Windshield heat capacit-y [J/°C] 

G; Windshield heat exchange coefficient, inner (cabin) side [W/°C] 
Windshield heat exchange coefficient, outer side [W/°C] 
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In this modelling approach, the windshield is just a "point" where all the glass 
mass is concentrated (with the heat capacity Q and whose temperature is 
This "point" exchanges thermal power with the ambient air proportionally to 
heat exchange coefficient G^, which is obviously dependent on car velocity. It 
exchanges also thermal power, proportionally to the heat exchange coefficient 
Gi, with what we have called "Inside air exchange temperature". The reason 
for this definition lies in the fact that, depending on the position of the air dis- 
tribution flap, the temperature to be considered for the heat exchange is dif- 
ferent. If the air distribution group sends the air to the windshield (defrost 
mode), must be the blown air temperature. When the air is blown from 
other outlets (i.e. floor or vent mode) one has to take into account cabin com- 
partment air. 

This model approach is necessary also because MRT sensor does not measure 
T^. As it is shown in [2], MRT is representative of a weighted mean of quanti- 
ties related to comfort. It takes into account both surface temperature and air 
temperature. Therefore, it can be used for: 

► Dew point temperature, in combination with relative humidity meas- 
urement. 

► when not in defrost mode. 

It is easy to understand that this is a simplified picture of a very complex phys- 
ical phenomenon. But we have to take into account that the final goal is to set 
up an algorithm that can run on a standard production control unit, therefore 
we have to keep it as simple as possible, and at the same time it has to reliable. 

Therefore, heat exchange coefficients cannot be calculated from literature for- 
mulae, but have to be identified thanks to an experimental methodology, keep- 
ing in mind that we have to simulate as the temperature of the part of the 
windshield where it is most likely that condensation takes place. From [1], this 
location is behind rear mirror (this has been confirmed also by testing per- 
formed at Centro Ricerche FIAT). 

From the above discussion, it is clear that this model approach must be cou- 
pled to a testing methodology and to an identification procedure so to find out 
the value of G^ and G^,, which are the unknown quantities that cannot be esti- 
mated a priori. 
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4 Testing Methodology and Identification Procedure 

The car has been equipped with the following sensors: 




Fig. 4. Car experimental set up. 

The first series of testing has been performed with the following goal: to iden- 
tify Gi end so to be able to reproduce, by means of the model, the time 
behaviour of T"^, as measured by the sensor placed behind the rear mirror. 

The car has therefore been placed in a climatic chamber where humidity and 
temperature can be controlled. The testing has been performed in the follow- 
ing way: 

► Ambient conditions (temperature and relative humidity) are fixed and 
kept constant throughout the testing. 

► Engine is turned on. 

► A value of air flow is selected. 

► Acting on air distribution flap, warm air is sent to the windshield (warm 
up period). 

► After the warm up period, acting again on air distribution flap, the air 
is blown through the front outlet (cool-down period). 
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When the cool-down period is over, another value of airflow is selected and the 
procedure is repeated until all airflow values of interest have been tested. 
Then, other values of external velocity and ambient conditions are imposed. 
The methodology has been optimised so that no more than two weeks of test- 
ing are necessary. The resulting time behaviour for measured is shown in 
the following figure, together with the corresponding simulated quantity: 




Fig. 5. Testing and simulation comparison. 

The simulated curve has been obtained by means of equation (1). An algorithm 
developed in Matlab-Simulink that minimizes the error between acquired 7”^, 
and measured one has identified values of end G^. Obviously, some of the 
tests are used for identification and some for validation. 

The above results show that cab be reproduced by means of model (1) in a 
reliable way. The other quantity that it is necessary to reproduce is the dew 
point temperature. This quantity is not simulated, but calculated from MKT 
and relative humidity sensor signals. Therefore, a series of validation tests for 
the whole approach have been performed. The comparison to be made is 
between the incipient fogging condition measured by the optic sensor and the 
corresponding value calculated by the algorithm, as shown below: 

The test has been performed on road under real operating conditions. It is easy 
to see that every fogging condition detected by the optic sensor is also detect- 
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ed by the estimation algorithm. Moreover, the estimation algorithm is always 
predictive with respect to the optic sensor, as expected. This allows to make 
the strategy more effective from an energetic point of view, since there is no 
latent heat to provide in order to prevent fogging. 




Fig. 6. Validation for incipient fogging detection. 



5 Potential for Integration with Other Climate Control Features 

As mentioned above, from cost benefit analysis point of view, it is very impor- 
tant to evaluate the potential for other functions. We have already mentioned 
that MRT sensor can be cost effectively used as feedback for the climate com- 
fort. MRT sensor combined with an externally controlled variable displace- 
ment compressor is an important Fiat development line in order to reduce the 
impact on fuel consumption increase given hy the air conditioning system [3]. 

In [3] it has heen shown that thanks to control and hy using this component, 
fuel consumption increase due to air conditioning system can he reduced up to 
50%. This comes from the fact that in most cases we can avoid reheat, just 
transferring to the air the right amount of refrigerant power that is needed to 
assure comfort. This implies more or less to have a control on the evaporation 
pressure level. 
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But if we can control this last quantity, this implies that we can vary (within a 
certain range) also the humidity content of the blown air. Obviously, we can- 
not humidify: but we can modulate the dehumidification of the air that it is 
treated by the evaporator. This offers a potential for cabin air humidity man- 
agement (it is clear that we cannot guarantee in all conditions that minimum 
impact on fuel consumption and desired humidity level are satisfied at the 
same time). 

With the sensor layout that we have described in this paper, this function can 
be assured without the need for additional sensors (obviously, provided that 
we have an externally controlled variable displacement compressor). This fea- 
ture has been implemented on a prototype, allowing the user to select both 
comfort temperature and humidity level. Since impact on consumption may be 
un important issue for mobile air conditioning systems in the near future, it is 
likely that externally controlled variable displacement compressor may dif- 
fuse. Therefore, humidity management might also diffuse. 



6 Conclusion 

We have shown that adding a relative humidity sensor to a MRT one, we have 
a reliable estimation of incipient windshield fogging condition. It is a predic- 
tive method: we get the information when condensation has already taken 
place, thus requiring more energy to remove it (because of phase change). It is 
not sensitive to dirt or to smoke. Moreover, there are already available on the 
market MRT sensors integrated with relative humidity ones. The calculation 
and testing necessary to implement this approach on a new car requires no 
more than two weeks. 

The proposed sensors layout provides signals both for climate control and for 
humidity management. With respect to standard layout, it allows a reduction 
in the number of sensors. Standard production FIAT Auto approach to guaran- 
tee the same functions (thermal comfort and incipient fogging estimation) 
requires 4 sensors: cabin air temperature, ambient temperature, solar sensor, 
optic fogging sensor. The one described here, 3 sensors: ambient air, MRT sen- 
sor, humidity sensor. It has to be taken into account that optic fogging sensor, 
besides being sensible to dirt, it is also quite costly. If we add also humidity 
management function, standard production layout would require an addition- 
al relative humidity sensor. As discussed above, this approach does not require 
additional sensors for humidity management. In this case, there would be two 
sensors less.This approach, thanks to its high level of integration, offers high 
potential for cost savings. 
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Abstract 

Next generation car architecture will be marked by seamless com- 
ponent integration. The „Mechatronics Concept" enables compos- 
able architectures in the car and will lead to increased development 
quality The upcoming of LIN™ and FlexRay™ based networking sys- 
tems in the car is in line with this new development paradigm, 
marked by model based system development, where the system 
design is supported by component modelling at each stage. This will 
lead to results which are „correct-by-construction" and will shorten 
the qualification cycle times concurrently with a significant increase 
in quality and reliability. Sensor design will have to comply with the 
new integration paradigms into the car electronic system. This will 
ultimately result into a software centric car architecture, which is 
being defined by the newly established AutOsar consortium. It will 
ultimately resolve the lifecycle disconnects between the semicon- 
ductor industry and the car industry and ensure a steady stream of 
innovation as well as assured spare parts supply. 



1 Vehicle Architecture 

1.1 Networking 

Figure 1 shows a typical network of a high-end passenger car. The powertrain 
bus is realized with a high-speed CAN bus with 500 kbit/s datarate, the multi- 
media communication is assumed through the MOST-bus and the Body-net- 
work is built by the fault-tolerant low-speed CAN running at 125 kbit/s. In this 
architecture the LIN-bus acts as a sub-bus below the module level depicted. 

In this architecture sensors are part of the functional subsystems like engine 
control, seat-control or navigation, and many more. In many control systems 
today, data is captured for the specific purpose of the subsystem. However it 
is much more practical, to make sensor data widely available. A good example 
is the speed information extracted from the four wheelspeed sensors of the 
ABS-system. In today's cars this speed information is distributed through the 
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in-vehicle network to the engine management, the dashboard, the radio, and 
many other systems in the car may pick-up this data. In the synergetic case, 
the ABS system dispatches the consensed speed information on the network 
for the benefit of all bus participants. 



Diagnostic 

I 



HS CAN 




Fig. 1. Example for a vehicle reference architecture. 

The architecture developments in the car have engaged into major network 
reworks: 

► In-vehicle networks are moving towards time-triggered architectures 
based on LIN™ and FlexRay™ for ease of system integration in a real- 
time signalling environment. 

► Fault tolerant systems are emerging, based on safe networking, leading 
to higher system availability. System requirements for x-by-wire appli- 
cations are being defined these days, early adopters will emerge on the 
market in 2006. 

► Decomposition of functional system blocks into smaller entities is ongo- 
ing. The governing factor is the mechatronics concept of distribution 
and standard interfaces, which leads to standardization at the module 
level. This concept is novel to the car industry, which is still marked by 
a high number of customized electronics today. 

► Concentration of computing performance in a few powerful nodes in 
the car is the architectural dream of the car industry today. In combi- 
nation with a strong network, functional computations can then be 
moved around the network as per resource availability and ease of sys- 
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tern partitioning. The AutOsar initiative of the car-OEMs launched in 
September 2003, has the goal to define the software interfaces to make 
this software exchange possible. 



1.2 Mechatronics 

Flexibility in software module exchange goes hand in hand with hardware 
compliance to specific interface standards. The mechatronics concept defines 
the interfaces of combined mechanical actuators or sensors together with its 
controlling electronics. This entity is then encapsulated and ready for use in a 
larger vehicle context of networked modules. In this sense the software 
exchange concept in definition with AutOsar is linked intimately to the mecha- 
tronics hardware approach based on standard interfaces. 

In this environment, components can then be sourced "Off-the-Shelf", discou- 
pling its technological development from the semiconductor growth pattern. 
This process is of utmost importance for the fast moving electronics industry, 
as well as the slower automotive industry. It solves the lifetime discrepancy 
problem between electronics and automobiles. In a well designed system, 
spare parts can be sourced out of current production rather than from shelved 
components. Under this regime, products don't have to be produced in obso- 
lete semiconductor lines or from components stored for many years under 
nitrogen any more. The mechatronics method is similar to the replacement of 
a light-bulb: old light bulbs are replaced with ones from current production, 
using newest technology rather than with stored bulbs, produced at the time 
when the car was build. 

The rapid electronic progress makes semiconductor components obsolete 
much earlier than the long lasting car requires, which typically are between 
15 and 25 years. In addition, the speed of innovation increases. In the mecha- 
tronics environment, new semiconductor technology can be applied faster to 
the application: as the mechatronics components have well defined bound- 
aries, new components can be incorporated in the vehicle much faster. The 
resulting steady introduction of new semiconductor technologies helps speed- 
ing up innovation in cars additionally. This industrial mechanism copies the 
momentum in place with the computing and networking industries today. 

However the status of mechatronic interfaces is not satisfactory yet. Although 
major interface standards exist on the signal level such as LIN™, CAN or 
FlexRay™, other mandatory interface standards are still desperately needed. 
Connectors exist in a large variety at different car makers, preventing signifi- 
cant exchange of components. On the same token mechanical fixtures are far 
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from being standardized. In total the mechatronic concept is still far away 
from reality, due to the fact that these simple, but important interface stan- 
dards are simply not existing industry-wide. Therefore the product example in 
figure 2, which is in production since a few years, will remain a single solution, 
if the industry cannot act quickly to set the mechanical requirements for 
mechatronics interfaces. 




Fig. 2. The mechatronics concept of distributed control. 



1.3 Software Integration 

When interfaces are fixed, the basic functions of the mechatronics components 
are too. In this context, innovation in functions is mainly driven through the 
software integration process. For this reason the interest in software develop- 
ment has drastically increased in the car industry over the last years. On the 
other side, in the mechatronics scenario, cost reduction is driven by produc- 
ing these components cheaper when using new innovative production tech- 
nologies or higher integrated electronic technologies. 

Therefore the highest amount of innovation is directly linked to the system 
integration process. Here the main charter is to compose new system func- 
tions out of existing mechatronics modules. Mastering network technology is 
at the source, but more important are all aspects of software portability, which 
are consequently addressed by the newly formed AutOsar-consortium, where 
the middleware linkage layer standards are in the process of being defined. 
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In this context the middleware can he seen as an additional software hus 
which allews sharing of resources through standard exchange mechanisms 
(figure 3). At the same time hardware abstraction occurs in support of the 
mechatronics concept, thus establishing the basis for software portability over 
different cemputing and I/O platforms. In practice it will net matter any mere 
if a PewerPC or a HC12, an ARM processor or any other controller cere exe- 
cutes the cede, as long as the timing and speed requirements are met. On the 
same token, bus access will be performed threugh standard routines, which 
will become part of the eperating system, very similar te what the VOLCANO 
system from VolcanoAutomotive is already offering today. 

A car system, eperating under a strong network regime, equipped with a mid- 
dleware system described above, will be marked by a drastic decrease of the 
number ef ECUs empleyed. This meets the requirements of the car makers, 
who must reduce the number of nodes open for after-sales service with seft- 
ware maintenance substantially, in order to meet the quality targets of these 
new cemplex software cars. 




NET Hardware BUS 



Fig. 3. Middleware concept for mechatronics. 



2 Sensor Technologies 

The progress ef MEMS surface micromachined semiconducter technolegy has 
led te rapid growth of the entire sensor market. The benefit of the semicon- 
ducter technelogy in this sector is clear: miniaturization means less material 
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cost. The progress of semiconductor technology is matched into this domain, 
linking the shrink roadmaps of the semiconductor industry into sensing, too. 
The compatible semiconductor manufacturing technologies lend themselves to 
further monolithic integration of MEMS devices with its associated data treat- 
ment circuits, including the prospects of further cost and size reductions, reli- 
ability increases and functional enhancements. For this reason we can expect, 
that silicon based surface micromachined MEMS-technologies will quickly 
replace all other known sensor technologies in the future such as bulk-silicon 
machining or otherwise mechanically treated materials. 



2.1 Motion Sensing 

Motion sensing has been pioneering MEMS technologies. Its functional princi- 
ple is a suspended polysilicon layer which acts as a capacitor. Its movement 
translates into a capacity change, which is being measured to yield the desired 
output of acceleration. 

Motorola has developed a dual-die solution mounted on the leadframe before 
transfer mold into a standard plastic package. The sensor is shielded by a 
glass-frit from exposure with the plastic during mold. The electronic circuit is 
placed adjacent to the sensor die on the lead frame and performs the data 
treatment and calibration of the capacitance measurement as a function of the 
acceleration. It is capable to generate a DC-voltage for electrostatic deviation 
of the measuring plate for diagnostics purposes. Final test can thus be per- 
formed without mechanical movements. 

The sensor is used in high quantities for airbag applications. Today, the output 
of the signal treatment circuit is ratiometric for further capture from an A/D 
channel of a microcontroller. In view of the anticipated architecture changes 
for stronger digital in-vehicle networking depicted above, digital interfaces are 
considered for future product derivatives. 



2.2 Pressure Sensing 

The underlying technology for pressure is similar to motion sensing. Through 
a hole in the package, the polysilicon layer is exposed to air-pressure (or other 
media), thus modulating the distance between another fixed layer according 
to changes in air-pressure. Again the transversal deviation of the material is 
first measured capacitively and then calibrated and amplified through an addi- 
tional circuit, which finally generates the signal output for further digital pro- 
cessing by a microcontroller. 
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2.3 Imaging 

Image sensing is moving rapidly towards static CMOS sensing, compatible 
with standard CMOS-technology, and away from dedicated CCD-technology. In 
combination with the available embedded computing performance in the 
1000 MIPS-range, real-time object recognition becomes feasible for automo- 
tive traffic situations. Again the link to standard semiconductor technology 
makes this technology increasingly cost effective for general use. 



2.4 RF technologies including Ultra-Wide Band 

Radio frequency technology is receiving a new impulse through the advent of 
high bandwidth Si-Ge technologies. The 77-GHz band is in reach of monolith- 
ic technologies and permits to cost-effectively set up RADAR systems for 
advanced cruise control systems. RADAR is able to reliably determine distance 
and relative speed of other objects on the road and can he used in conjunction 
with imaging to strengthen the signal capture process for the car in traffic. In 
addition the traditional function of RF as a transmission medium for commu- 
nication channels will rapidly increase once local car-to-car communication 
will receive attention for increasing road safety. Derived from the spread spec- 
trum technology, broad GHz-bands are being assigned in some regions of the 
world for the two major applications: data transmission and Radar. The advan- 
tage of Ultra Wide Band (UWB) is that it can be configured for ultra-low power 
when only small transmission bandwidth is required. 



2.6 MEMS based Initiators 

There are numerous other applications for micromachining technologies. 
Today's airhag initiators are using HfHx-deposition aligned with semiconduc- 
tor process technology. The integration of the initiator, with its driving elec- 
tronics follows the mechatronic systems trend. 



2.6 Data Fusion 

Kalman filtering is an old concept to extract system state variables through 
measurement of peripheral system values and its transformation with esti- 
mated system properties. The same can he applied in modern cars. The many 
different measured variables can be transformed into a rugged view of the 
complete system, where the internal state variables are the result of a con- 
sensus process. All which is required is the knowledge of the system signal 
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transfer properties and enough computing performance. As such, the capabil- 
ity is there today, and it should be emphasised that the massive increase of 
computing resources in the car will ease data fusion as a means of generating 
further internal signal data points. 



3 Ambient Intelligence 

With the availability of pervasive sensor data in the car, a new phase is about 
to start in automotive electronics. In combination with networking and mas- 
sive computing resources, new systems are in definition which make traffic 
inherently safer, make travel times more predictable and driving more eco- 
nomic. Public authorities have recognized that the high number of fatalities on 
European roads must be reduced and have discovered that electronics still 
bears a largely underexploited potential. For this reason e-Safety initiatives 
concentrate on the capture of in-vehicle data as well as surrounding environ- 
mental data. Connectivity between cars and share of captured data uses the 
network effect of multiple data sources, leading to very economic safety sys- 
tems in the car. The problem to be solved is not of technological nature, but 
key to success and widespread deployment is interface standardization. I.e. in 
which format data is to be shared between cars. In the jungle of multiple 
exchange mechanisms already introduced today, this is a very difficult prob- 
lem to solve, especially as each proposed method has serious technical impli- 
cations for the companies driving their preferred solutions. The reward how- 
ever is big for the society: a phenomenal increase in transportation safety for 
the consumers and a new surge in semiconductor demand for the participat- 
ing electronics suppliers. 



4 Integration Process 

The model based development concept is taking shape in the market place. It 
acknowledges the importance of the product design process for further 
improvements in product and system quality and places the focus on shifting 
development efforts away from the qualifying back-end into the "correct-by- 
design" front-end. However this requires a massive shift in development 
methodology through concurrent modelling and simulation efforts. The avail- 
ability of models before the validation phase, will incur a major shift in the 
industry and can only be achieved as a concerted activity between car-OEMs, 
as well as its Tier-1 and Tier-2 suppliers. 
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4.1 Increasing product quality 

The car electronic reports high component quality between 1 ppm and 
10 ppm. Typically its component quality is higher than for other commercial 
industries, namely consumer and computer. However due to the lasting nature 
of cars and because failure of electronics is strongly impacting the end cus- 
tomer, quality needs to be improved beyond this level. Today's quality system 
relies on certification procedures at the end of the development cycle through 
summer- and winter-testing in the car. After this, the production process must 
be "frozen" to maintain this certified production process. The result is a 
rugged, but inflexible production process, with significant cost associated. 

To go beyond this level, significant changes need to happen: 

► The manufacturing volumes need to be increases by another order of 
magnitude. Serious learning of quality defects can only happen with 
volumes beyond 10 Mio pcs per year. For a 1 ppm failure rate, this 
means for this example that only 10 failing pieces are generated per 
year, which is not enough for deep rooted analysis. 

► Quality is a component property, which must be designed-in rather 
than tested-in. This requires a paradigm shift in development, where 
certified development processes are used in the future (c.f. ISO-61 508) 

► On-Line component diagnostics need to be improved further. As the 
component is in use in the application on-line testing will assure that no 
degradation of system properties will occur. This property must be 
designed into the component. An option is to switch the component to 
service mode at specific predefined idle times. 

► The software development and integration process must become as 
rugged as hardware design, where 1 Mio lines of VHDL-code can be 
handled with no failures. The good results in hardware design are due 
to the rugged encapsulation and reuse standards, which must be 
applied in a similar fashion to software. 

► Encapsulation of generic (soft- and hardware) modules which can be 
qualified independently from the car will become a must do! 

These are revolutionary changes in the way we write software and the impli- 
cations are serious. However, it is well recognized, that the soft and hardware 
integration process must be improved significantly, or otherwise the product 
quality cannot be mastered in the next generation of complex automotive elec- 
tronics. 
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4.2 Encapsulation 

Mechatronics means encapsulation: A specific function is captured in a "box" 
and described through its interfaces. This is true for a mechatronics stepper 
motor with LIN-Bus capability as well as for a bus-capable temperature sensor. 
The same concept applies to embedded software in the same way. Today, UML 
is gaining acceptance in the automotive industry in order to increase reuse of 
existing code and master the complexity better through clean partitioning and 
solid description of the interfaces. Together with clean program management 
tools, the development process becomes more reliable and straight-forward 
than before. It aligns with IEC-61508 and lends itself easier for certification of 
safety critical components. 

As a result the development resources can be shifted away from the "back- 
end", where product system integration testing, qualification and reliability 
engineering is done, towards the well-ordered development front-end. 

This new design concept supports virtual prototyping and simulation, when 
carefully tailored interface layers separate the system into simulatable parti- 
tions. Practically, this means, that the industry will deemphasize investments 
into quality systems like QS9000 (now integrating into TS 16949) and shift this 
efforts into design quality systems such as CMMI. In conjunction with a 
rugged and well followed development process, quality of the results can be 
guaranteed as well as project timeliness and budget compliance. 



4.3 Tools 

Encapsulation means increased model based developments, where for every 
encapsulated module we have the following properties: 

► A full executable specification exist for every encapsulation, this can be 
a 'C'-model or Matlab model or any other executable description. 

► A model of the environment is existing for the encapsulated block, 
preferably in the same description language. 

► Simulation capability exists to exercise the model, which is virtual for 
hardware and "real" for software, to ensure operation. 

The test-suite is defined and guarantees operation and qualification. The goal 
is to be able to qualify encapsulated component within its defined boundaries. 
With this, a car development becomes a layered approach with the primary 
objective of interface design, definition of the control hierarchies, and specifi- 
cation of the functionalities of the encapsulated module. 
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A large variety of development tools is on the market. An attempt is made 
here to order those into major categories. In brackets are some examples of 
existing tools and vendors: 

► Requirements Management (Doors) 

► Configuration Management (Clearcase) 

► Defect Ttacking (Clearquest) 

► Process Flow Control (Rational Unified Process) 

► Software Architecture (UML) 

► Component Design: Case tools and Synthesis (Matlab Simulink, 
Stateflow, Veristate) 

► Network Design (Volcano-Automotive, Vector, IHR) 

► Simulation & Modelling Framework (System-C) 

► Unit and Integration Testing (LDRA, PXI) 

► Debug and Calibration (Metrowerks, Windriver, Lauterbach, 
PowerNexus) 

► In essence a development team has to make a choice for a specific tool 
suite and work on the tool-interfaces, as a completely seamless envi- 
ronment is not yet in existence today. 



5 Summary and Outlook 

We have to recognise that the semiconductor microsystem technologies have 
led to a big increase of sensor data captured in the vehicle, making cars safer, 
more reliable and of higher use-value. This trend will continue as long as the 
sensor technology can profit from the tremendous semiconductor productivi- 
ty increase, according to Gordon-Moore, which we continue to observe. 

As an additional element of growth, the sensor data distribution will reach 
beyond the network of the vehicle. Progress in wireless technologies will ease 
capturing of data from further sources not accessible until now. Especially the 
treatment of environmental data around the car will revolutionize car safety 
systems. Data will be sourced from other cars, from sensors on the road infra- 
structure, or from a central server. This can be information about the actual 
road conditions, weather influences or particular traffic situations. 

With cheap communication technology available, mobile ad-hoc networks will 
form on the road and between traffic participants to transmit pervasive sensor 
data through the system. New applications based on this concept of "Ambient 
Intelligence" will open the next chapter of exponential growth for microsys- 
tems and with this will generate a massive pull for new embedded semicon- 
ductors in the car. 
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To enable the integration process of Sensors in the network, encapsulation will 
be of utmost importance to comply with the mechatronics concept of automo- 
tive electronics architecture. General availability of sensor models will change 
currently existing development paradigms towards a model based automotive 
development system. 
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Abstract 

Modern in-vehicle telematics systems typically integrate a broad 
range of entertainment features, like AM/FM radio, TV, and CD/MP3/ 
DVD players, with advanced navigatien and communication features 
and present them te the user through a consistent automotive-ade- 
quate interface. In the future, passengers will be able to access and 
control these features independently through their own consoles. 
For the required high-speed autometive-grade internetworking, the 
industry has established the MOST (Media Oriented Systems 
IFansport) standard. This standard specifies the physical layer up te 
the network management layer tegether with a set of application 
oriented device interfaces. For distributed applications, however, the 
standard lacks an appropriate middleware layer. HAVi (Home 
AudieA/ideo Interoperability) is a carefully designed middleware 
standard that allews IEEE 1394 (Firewire) based devices to interop- 
erate. In this paper, we shew that HAVi can be adapted to the MOST 
bus. In particular, we describe in detail the essential changes to the 
HAVi standard that are required to use HAVi as a middleware layer 
on top of MOST. 



1 Introduction 

The automotive industry is currently in a transition phase from being hard- 
ware focused to being seftware driven. A medern premium vehicle is a cem- 
plex distributed software intensive system on wheels involving several million 
lines of code and interconnecting as many as 77 central units by means of a 
hierarchy of domain-specific heterogeneous sub networks (see also [3]). The 
challenges faced when engineering these advanced vehicles rival these of 
modern airplanes. One sound approach to meet these challenges is te develep 
middleware standards that provide a stable pregramming interface for appli- 
cations. One example is the OSEKA7DX family of standards for an open-ended 
architecture fer distributed central units in vehicles [9]. 
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In-vehicle telematics systems are themselves sophisticated distributed soft- 
ware intensive systems (cf. figure 1). They typically integrate a broad range of 
entertainment devices, like AM/FM radio, television, and CD/MP3/DVD play- 
ers, with advanced navigation and communication features and present them 
to the user through a consistent automotive-adequate interface. In the future, 
passengers will be able to access and control these features independently 
through their own control units. For the required high-speed automotive-grade 
internetworking, the industry has established the MOST (Media Oriented 
Systems TTansport) standard [6]. The MOST bus provides the means for trans- 
mitting control commands, asynchronous messages and synchronous multi- 
media streams with an overall bandwidth of 25 Mbit/s. It interconnects con- 
trolled (e.g. a CD changer) and controlling multimedia devices (the head unit 
and rear seat control units). The standard specifies the physical layer up to the 
network management layer together with a set of application oriented device 
interfaces. For distributed applications, however, the standard lacks an appro- 
priate middleware layer including higher-level management services (such as 
resource management, event management, stream management, or a general 
bus-independent messaging). These services typically would have to be devel- 
oped in a proprietary manner. 




Fig. 1. MOST based telematics ring in the Maybach. 
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HAVi (Home AudioA/ideo Interoperability) is an existing carefully designed 
middleware standard that allows IEEE 1394 (Firewire) based devices to inter- 
operate [1, 10], In this paper, we show that HAVi can be adapted to the MOST 
bus. The advantages are twofold: first, we cleanly separate the application 
logic from communication / transport / media access aspects, and, second, we 
reuse the system management services of the HAVi middleware. We summa- 
rize the essential changes to the HAVi standard that are required to use HAVi 
as a middleware layer on top of MOST. These changes are documented in detail 
in a draft specification [11]. The guiding principle has been to follow HAVi's 
design principles and to keep changes as minimal as possible. Therefore, these 
changes could be easily integrated into a future version of the HAVi standard. 

One of the original design goals of HAVi has been to be transport layer inde- 
pendent. The present HAVi specification, however, does not meet this goal 
completely. IEEE 1394 specific issues show up in layers that should in princi- 
ple be shielded from them such as addressing, stream management, and 
device discovery. One challenge of adapting HAVi to MOST therefore was to 
identify these areas and to clearly factor out transport layer specific aspects. 
This is a further - more general - contribution of our work and could be used 
to develop a future transport layer independent version of the HAVi standard. 

The remainder of this paper is structured as follows: The next section shortly 
describes the HAVi standard and how it could be applied in the automotive 
environment. Section 3 describes the MOST bus and the changes necessary to 
extend the HAVi standard to support the MOST bus. In section 4 related work 
is discussed while section 5 concludes the paper. 



2 Utilizing the HAVi Multimedia Framework in the Automotive 
Environment 

The HAVi architecture (defined in [1]) has been developed to allow for an inter- 
operability and extensibility of devices in the home entertainment environ- 
ment, which is in many aspects similar to the entertainment networks in vehi- 
cles. In this section we first describe the HAVi framework and give examples 
of possible uses in a vehicle entertainment system. We then discuss changes 
necessary to the HAVi standard because of the special requirements of the 
automotive environment. 
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2.1 The HAVi Standard 

The HAVi standard specifies a set of distributed management services and 
their interfaces as required in the home entertainment environment, for exam- 
ple for the management of video and audio streams. Furthermore, a controlled 
device in the HAVi environment may provide its own device driver (called a 
Device Control Module, DCM), which can be executed on a controlling device, 
and its own user interface (in a so-called Havlet). This concept makes a HAVi 
network extensible, because it allows to add previously not supported con- 
trolled devices. 

An important aspect of the HAVi architecture is the distinction between dif- 
ferent types of devices, which allows to have complex controlling devices (e.g. 
set top boxes) as well as more simple and therefore cheaper controlled devices 
(e.g. a CD player). Additionally, this concept allows to integrate legacy devices 
that are not aware of HAVi. Both is important for the automotive environment, 
as the former allows to include more of the numerous often simple Electronic 
Control Units (ECUs) in the HAVi network and the latter because it allows a 
reuse of existing devices, reducing the transition costs. 

The HAVi architecture defines the following device categories: 

► Full AudioA/ideo device (FAV): FAVs are controlling devices that run all 
HAVi managers (see below) and are able to execute the DCMs for con- 
trolled HAVi devices (BAVs and LAVs). To run an uploadable DCM pro- 
vided by a BAV, an FAV has to contain a runtime environment for Java 
byte code. Applications using the HAVi framework will be executed on 
an FAV or lAV The head unit of a vehicle telematics system would be an 
FAV. 

► Intermediate AudioA/ideo device (lAV): An lAV is similar to an FAV but 
has a reduced set of managers and no runtime environment for Java 
byte code. This allows for cheaper controlling devices with less 
resources. In the automotive domain, a rear-seat unit could be realized 
as an lAV. 

► Base AudioAfideo device (BAV): BAVs are controlled devices that provide 
HAVi-specific information and a DCM code unit as uploadable Java byte 
code in its ROM. An FAV can download this code unit and run the DCM 
for a BAV. 

► Legacy AudioA/ideo device (LAV): Existing controlled devices that are 
not aware of HAVi can be included in a HAVi network as LAVs. An FAV 
or lAV can provide an embedded code unit and run a DCM for such a 
device (e.g. a device supporting the IEEE 1394-based Audio/Video 
Control protocol, AV/C). Existing controlled MOST devices, such as a CD 
changer or a radio tuner, will have to be treated as LAVs. 
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The management components defined by the HAVi architecture are shown in 
figure 2 for a HAVi FAV device. The Communication Media Manager for the 
IEEE 1394 bus (1394-CMM) provides an abstraction for sending and receiving 
data via the IEEE 1394 bus. The Messaging System provides a higher-level 
communication mechanism where applications and managers on different 
EAV/IAV devices can communicate either by exchanging messages or by using 
a Remote Procedure Call mechanism called HAVi Remote Method Invocation 
(RMI). 







Applications F 









Interoperability API 









Platform-specific API 



Vendor-specific Platform 



Fig. 2. Components of the HAVi architecture. 

With the Registry all applications and services (called Software Elements) have 
to register with describing attributes. Usually the telematics system of a vehi- 
cle comes in many different variants, for example with or without integrated 
mobile phone. An application will therefore use the Registry to determine 
what devices and services are avtiilable in the system. The Event Manager is 
responsible for propagating system events, indicating for example a network 
reset or new and gone devices, in the HAVi network. Connections for trans- 
mitting multimedia data between devices can be set up and controlled through 
the Stream Manager. In a vehicle telematics system multiple streams, for 
example for radio, telephone as well as warning signals, will have to be coor- 
dinated. The Resource Manager is responsible for managing conflicting access 
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to a resource of the HAVi network. Using the Resource Manager an application 
can reserve a certain resource or start a negotiation process, if a resource it 
wants to use is already occupied. The Resource Manager would be used in a 
telematics system to manage the conflicting access for example to a CD chang- 
er from front- and rear-seat control units. 

Each device in a HAVi network is represented by a Device Control Module 
(DCM). The DCM for an FAV or lAV runs on the device itself, the DCM for a 
BAV or LAV device on a suitable FAV or lAV A DCM for an LAV device can 
communicate with it either via a proprietary protocol over the IEEE 1394 bus 
or even using a separate communication mechanism. A DCM provides a 
Functional Component Module (FCM) for each functionality of a device. For 
example, a TV system that also includes a VCR would provide two separate 
FCMs. The HAVi standard defines interfaces of a number of typical devices for 
the home entertainment environment like a VCR or an amplifier. For the auto- 
motive domain suitable DCMs and FCMs will have to be created, which allow 
to access the functionality of telematics devices on a higher level of abstrac- 
tion as compared to using MOST mechanisms. The DCM Manager, finally, is 
responsible for finding and installing the appropriate DCM for each BAV and 
LAV device on a suitable FAV or lAV in the HAVi network. 

Together the interfaces of the managers and the standard interfaces of DCMs 
and FCMs make up the Interoperability API of the HAVi architecture, based on 
which portable applications can be developed. 



2.2 General Extensions for an Automotive HAVi 

As HAVi has been designed for - and is well suited to - the home environ- 
ment, some modifications and extensions are becoming necessary for the use 
in the automotive environment. While the basic requirements at home are 
easy installation and use, in a vehicle they are system stability and operation 
in rough environments. This leads to the following extensions of the HAVi 
specification, which are discussed in more detail in [8]. An important guideline 
for the design of these modifications was the downward compatibility with the 
existing HAVi specification and devices: 

► Power Consumption Management - differently from the home envi- 
ronment, the power consumption of the devices is a crucial issue in 
automotive networks. We therefore propose to add a network-wide 
power management to the HAVi architecture. It consists of a Power 
Management Server (PMS), that operates as a HAVi manager, and a 
number of Power Management Clients (PMCs) representing the HAVi- 
based devices in the network. Every decision on changing from normal 
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activity to a lower activity level (as down to deep sleep mode) is taken 
by the server and can be based on a large variety of information sources 
and pre-configured scenarios. 

► Network-wide Error Handling - to cope with the increased demands 
on system stability, an error handling will have to be added, which has 
far-reaching capabilities to detect errors and to react accordingly 
Similar to the power management, the error management has one cen- 
tral element - the Error Management Server (EMS) and distributed 
Error Management Clients (EMCs) that support and inform the EMS to 
detect errors and to perform selected exception handling mechanisms 
locally. 

► Smart Device Access Management - very important in distributed 
networking is the management of competing device or function access- 
es (i.e., two applications residing in the network try to access the same 
network resource). HAVi has sophisticated means to deal with these 
scenarios, but relies heavily on user decisions to resolve access conflicts. 
In a vehicle this is not acceptable because of driver distraction and will 
have to be replaced by a mechanism, which considers applications' pri- 
orities. Based on these priorities an automotive HAVi can ensure deter- 
ministic access for multiple applications to the same resource. 

► Smart Bandwidth Management - having the access to a certain 
resource might not be sufficient, if the required data transmission rate 
is not available. As in HAVi multimedia streams are already controlled 
through the Stream Manager, this component can be modified to take 
into account the same priorities as for the device access management 
and to release or to suspend connections accordingly. 

As mentioned before, HAVi provides standardized FCM interfaces for the most 
common home devices. Because vehicle telematics devices are very specialized 
and differ from home devices, FCMs for common vehicle devices need to be 
added. The following shows some examples for automotive FCMs, which will 
be based on the corresponding MOST function blocks to a large degree: 

► AV Disc Changer FCM - extends the standard AV Disc FCM with the 
yet lacking functionality to change the active disc. 

► Navigation FCM - provides access to the navigation unit, for example 
for entering a destination automatically. 

► Automotive Timer FCM - comprises support for traffic announcements 
and other automotive specific needs for tuner applications. 

► Mobile Phone FCM - will allow the control of a mobile phone. 
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3 Adapting the HAVi Framework to the MOST Bus 

In the previous chapters we have discussed the requirements of the automo- 
tive domain regarding a middleware for its multimedia applications. We have 
also shown how the HAVi framework fulfils these requirements. In this chap- 
ter we discuss the changes necessary for adapting the HAVi framework for use 
with the MOST hus. Because the MOST bus is currently coming into use in 
vehicles such as the Audi A8, the BMW 7er series and the Mercedes E-Class, 
adding support for MOST to the HAVi standard is crucial for its use in the auto- 
motive environment. 



3.1 The MOST Bus 

The MOST bus (as defined in [6]) provides the means for communication 
between controlling and controlled multimedia devices and is able to transport 
multimedia data in a synchronized way. It has been designed especially with 
the requirements of the automotive environment in mind. Because the MOST 
bus transmits data in a synchronous way and because of its low susceptibility 
to electro-magnetic interference, it can provide the required reliability. 

The MOST bus is structured in a ring topology. In its current version, it sup- 
ports up to 64 devices and provides an overall available bandwidth of 
24.8 Mbit/s. It uses plastic optical fiber (POP) as its connection technology, 
which results in a reduction of the problems caused by electro-magnetic inter- 
ference, but is more robust than glass optical fiber. 

For transmitting data, the MOST ring supports the following types of data 
channels with different characteristics: 

► Control channel: The control channel is used to send control messages 
to a device in order to initiate a certain functionality (e.g. starting to 
play a certain track of a CD) or to retrieve information about its current 
state (e.g. the track position). To this end, control commands and their 
parameters as well as the device's responses or periodic status reports 
are exchanged via control messages. The control channel provides a 
bandwidth of 705.6 kBit/s. 

► Synchronous channel: For transmitting multimedia information 
between devices in a MOST ring, the MOST bus supports up to 60 phys- 
ical channels, where a physical channel corresponds to one byte at a 
rate of 44,1 kHz. Physical channels can be grouped to form a logical 
channel; for example, it takes four physical channels to make up one 
stereo audio channel of CD-quality. The MOST bus can therefore sup- 
port a maximum of 15 such stereo channels or 21.2 Mbit/s. 
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► Asynchronous channel: The asynchronous channel is intended for 
transmitting larger amounts of data in a more hurst-like manner. Access 
to the asynchronous channel is provided through a token mechanism. 
Of the maximum number of 60 physical synchronous channels (see last 
point), the capacity of between 0 and 36 can be allocated to the asyn- 
chronous channel. This allocation can be parameterized by setting a so- 
called boundary descriptor according to the needs of the applications. If 
the full complement is allocated - leaving 24 physical synchronous chan- 
nels - 12.7 Mbit/s are available on the asynchronous channel. 

The distinction between the control and the asynchronous channel for data 
communication allows for simple controlled devices (such as a CD player) that 
do not support asynchronous communication, making hardware requirements 
for these devices less expensive. 

Besides the low level communication mechanisms, the MOST bus also provides 
mechanisms for controlling the functionality of devices in a MOST network. A 
certain functionality is encapsulated in a function block, whose purpose can be 
compared to that of a HAVi FCM. The AudioDiskPlayer function block, for 
example, represents the CD player of the vehicle's sound system. Via the 
MOST bus an application can control the function blocks available in the MOST 
ring and receive information about their status. MOST specifies standard func- 
tion blocks for accessing typical functions of an automotive telematics system 
such as a CD player, an amplifier or a mobile phone. 



3.2 Main Differences between HAVi and MOST Mechanisms 

Because the HAVi architecture is influenced by the underlying IEEE 1394 bus, 
there are the following differences to the mechanisms of the MOST bus, which 
will have to be addressed for an integration of support for MOST 

► As the network environment in a vehicle is usually static, the MOST bus 
only uses local addressing schemes (16 bit) based on a device type spe- 
cific address (logical node address) or the position of the devices in the 
ring (node position address). There is no global unique addressing 
scheme similar to the global unique identifiers (GUIDs) used in an IEEE 
1394 network. Additionally, addresses may change when the configu- 
ration of the ring is changed. 

► Unlike the IEEE 1394 bus, where control messages are sent as regular 
messages of a special format, the MOST bus has a separate control 
channel for sending control messages with its own message format and 
mechanisms. 
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► The management of multimedia streams is more simple in the MOST 
bus as compared to the IEEE 1394 bus. The sources and sinks of a 
device are accessed only through the corresponding function blocks. 
HAVi on the other hand, makes a distinction between FCM plugs and 
device plugs and allows for internal connections. Also, in the MOST bus 
it is not possible to control the properties of a stream dynamically. 
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Fig. 3. Changes to the components of the HAVi architecture (compare to 
figure 2). 

To address these issues the following changes (cf. figure 3), have to be made 
to the components of the HAVi architecture. All other components not men- 
tioned here are not influenced by the extension: 

► The most noticeable change is a new MOST Communication Media 
Manager (MOST-CMM) that encapsulates all access to the MOST bus (i.e. 
all other components described below use the MOST-CMM to access the 
MOST's functionality). The MOST-CMM is described in detail in the fol- 
lowing sub section. 

► The interface of the Messaging System does not need to be changed. 
However, the Messaging System has to detect if a device in the MOST 
network is accessed and to call the MOST-CMM instead of the 1394- 
CMM. In a network which contains both a MOST and an IEEE 1394 net- 
work, the Messaging System additionally has to provide the functional- 
ity of a gateway. 
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► The functionality of the Stream Manager is based on the lEC 61883 pro- 
tocol extension of the IEEE 1394 bus. The functionality of the Stream 
Manager therefore has to be adapted to take into account the different 
(more simple) stream management capabilities of the MOST bus. 

► A Device Control Module (DCM) and its associated Functional 
Component Modules (FCMs) control the access to the functionality of a 
certain device. They therefore have to implement the device specific 
functionality of automotive MOST devices. 

► Finally, the Device Control Module Manager has to be extended slight- 
ly to be able to detect MOST devices and to find and install suitable 
DCM code units. 



3.3 The MOST Communication Media Manager 

The MOST Communication Media Manager (MOST-CMM) is the only new com- 
ponent that has to be added to the HAVi architecture for a support of the 
MOST bus. We will therefore describe it here in more detail. 

One important responsibility of the CMM is the mapping of the QUID address- 
es used in HAVi (which every IEEE 1394 device already provides) to the 
addresses used in the communication system. As the MOST bus provides no 
globally unique addressing scheme, we have to map MOST addresses to appro- 
priate HAVi GUIDs. Based on this address the MOST-CMM in a gateway device 
has to be able to decide, to which attached communication bus a certain mes- 
sage has to be routed to. 

We propose a simple solution (see figure 4), where the GUID begins as before 
with a leading OUID (Organizational Unit Identifier) describing the vendor 
who manufactured the device. To guarantee that an automotive GUID is dif- 
ferent from the GUIDs of existing IEEE 1394 devices, a vendor that produces 
devices for the automotive environment will have to request a separate OUID 
for this purpose. 

The remaining field of the GUID begins with a 1 byte identifier of the sub net- 
work, called the Netid. This identifier is different for each communication sys- 
tem attached to a gateway device. During initialization the Netid is 0 (which is 
also the default value, when no gateway functionality is required) and is then 
set by the gateway device appropriately. The next byte differentiates between 
MOST devices that are aware of HAVi (0x00) and devices that are not (0x01). 
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A MOST device that is HAVi-aware will provide a vendor-wide unique serial 
number, which constitutes the remainder of the GUID. The GUID for a MOST 
LAV device (which is not HAVi-aware) will contain its logical node address in 
the last two bytes. Finally, a further byte, the Busld, can be used to differenti- 
ate between different MOST buses that are directly connected, for example if 
there is a separate MOST bus for the front- and the rear-seat entertainment 
system. 
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Fig. 4. Addressing scheme for MOST devices. 

As mentioned before, we decided to encapsulate all access to the MOST bus in 
the MOST-CMM. This is somewhat opposed to the original HAVi specification, 
where higher level components need to have knowledge about IEEE 1394 spe- 
cific protocols. The Stream Manager, for example, uses the lEC 63881 protocol 
[2] to manage multimedia connections. Therefore, besides an interface for 
sending and receiving data, the MOST-CMM also provides interfaces for 
retrieving information about devices and for managing multimedia streams: 

► Retrieving device information: Besides the method CmmMost:: 
GetGuidList, which enumerates the GUIDs of all devices in a MOST net- 
work, the MOST-CMM offers the method CmmMost: :GetDeviceInfo to 
retrieve information about a MOST device, including vendor, product 
name and a list of the function blocks the device implements. 

► Asynchronous messaging: We decided to use the asynchronous chan- 
nel of the MOST bus to transmit HAVi messages (including HAVi RMI), 
as it is best suited for the messaging style of communication and unlike 
the control channel provides the necessary bandwidth. To this end, the 
MOST-CMM offers the method CmmMost: :SendPacket and an appropri- 
ate indication mechanism similar to the original Cmml394::Write. 
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► Controlling MOST devices: While asynchronous messages can be used 
for the communication between FAVs/IAVs, a different messaging 
mechanism is required to control existing MOST devices, which has to 
be done by sending control messages over the control channel. The 
MOST-CMM therefore offers similar methods as for asynchronous mes- 
sages for sending and receiving control messages. 

► Managing synchronous connections: Finally, the MOST-CMM supports 
various functions to manage synchronous connections and to query 
their current state. With CmmMost:: Allocate a certain source plug of a 
MOST function block can be caused to allocate a synchronous channel 
and with CmmMost: iConnect a sink plug to connect to this channel. The 
available bandwidth on the MOST bus can be queried with CmmMost:: 
AvailableChannels, information about the sink and source connectors of 
a certain MOST function block with CmmMost: :GetSyncDataInfo. 



3.4 Stream Management 

Changes to the HAVi Stream Manager are necessary, because the stream man- 
agement is more restricted in the MOST bus than with the IEEE 1394 bus. The 
main conceptual differences are as follows: 

► While HAVi and IEEE 1394 make a distinction between device and FCM 
plugs, the MOST bus only supports connectors for function blocks, sim- 
ilar to FCM plugs. To remain compatible, a DCM for a MOST device will 
therefore support a virtual device plug for every FCM plug of its associ- 
ated FCMs. 

► In IEEE 1394 it is possible to connect FCM plugs on the same device 
without establishing a connection over the communication system. In 
the MOST bus, however, there is no general mechanism to establish 
such internal connections. 

► In the MOST bus, the characteristics of a stream are defined by its 
source connector. It is not possible to set properties of the stream, like 
bandwidth, data type etc., dynamically. This restriction also has to be 
reflected by the Stream Manager. 



3.5 Device Control 

During start-up of a HAVi network, the DCM Manager scans the network for 
controlled BAV or LAV devices. For IEEE 1394 devices, all except LAVs can be 
identified through the information in their SDD ROM. The next step is to find 
an appropriate DCM for each of them. For a BAV this is done by uploading the 
DCM code unit either from the device or from a location it specifies. For an 
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LAV the DCM code unit has to be provided by the controlling FAV/IAV itself. If 
more than one FAV is able to run the DCM for a certain device, they initiate a 
negotiation process to determine the most suitable one. 

If an FAV/IAV also supports MOST devices, it has to perform the same proce- 
dure for the devices in an attached MOST network. For a HAVi-aware MOST 
device, the function of the SDD ROM is supplied by a special MOST function 
block called HAVi (with the tentative function block identifier OxDO). The 
Device Manager will then check if any of its embedded DCM code units 
matches a detected device and install this code unit as for an IEEE 1394 device 
(if necessary after an appropriate negotiation process). 

In an object-oriented HAVi stack (e.g. the FireBus stack from VividLogic [7]) a 
concrete DCM or FCM is often derived from a generic parent class, called for 
example GenericDcm or GenericFcm, which provides general functionality, 
like registering the software elements with the Registry. For the integration of 
MOST devices in HAVi, we propose to encapsulate the generic functionality of 
the MOST bus in a second abstract layer consisting of the classes MostDcm 
and MostFcm. These are intended to make the development of MOST DCMs 
and FCMs considerably easier by already including generic functionality, 
which any MOST device has to implement according to the MOST standard 
(e.g. returning the product or company name). A specific DCM or FCM derived 
from these generic classes only has the responsibility to implement the special 
functionality of the device, for example that of a TV tuner. It can keep the 
generic functionality provided by the more abstract classes or extend it to be 
more specific, where necessary. 



4 Related Work 

Today, the different manufacturers of vehicle telematics components use pro- 
prietary software architectures (such as Siemens VDO's Top Level 
Architecture, TLA) to implement the devices' functionality. Because the archi- 
tectures of devices from different suppliers have no common understanding of 
distributed management functions, they can only interact on the level of 
MOST mechanisms. As mentioned in the introduction, this means that few 
higher level management functions (like resource management) can be reused 
between following generations of a telematics system. Instead, they have to be 
defined and implemented separately for each generation. 
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Another standard that we have considered for connecting vehicle telematics 
devices in a flexible and extensible manner is Microsoft's Universal Plug and 
Play (UPnP) [12]. As the communication in UPnP is based on TCP/IP, it is more 
independent from the underlying communication network than HAVi. In UPnP 
device interaction is handled in a peer-to-peer fashion. The services offered by 
a device (actions or events) are described in a general way by a document of 
XML format. A service announces itself in the network by means of a service 
advertisement sent via multicast on a well-known address with a limited time 
to live (UPnP proposes a time to live of 4 hops). With a similar mechanism, an 
application can query for offered services through a service request. For the 
control of a service, control commands and events are exchanged via the 
Simple Object Access Protocol (SOAP). However, the mechanisms of UPnP are 
more suited for a flexible environment as compared to the more static envi- 
ronment in a vehicle. As yet, UPnP provides no mechanisms for a distributed 
management of resources. Interfaces to specific services are just beginning to 
be defined. 

A further standard which defines the interaction of controlled and controlling 
devices on a higher layer of abstraction is Jini [13], which is based on Sun's 
Java mechanisms. Here devices announce themselves to a special Jini Lookup 
Service in their vicinity, again through a multicast with a restricted time to 
live. Applications query a nearby Lookup Service, found through a similar mul- 
ticast-based mechanism, for a service they want to use. Services have to pro- 
vide a proxy as Java byte code, which handles all interaction with the service 
and which can be retrieved through the Lookup Service. An application uses 
this proxy, which can be compared to a DCM code unit in HAVi, to access the 
service. As with UPnP, Jini is more suited for a flexible environment and does 
not provide the distributed management services required for the automotive 
environment. 



5 Conclusion 

In this paper, we presented the case for employing a multimedia framework to 
achieve extensibility and reusability of software in the automotive telematics 
domain. We showed the domain-specific requirements for such a framework 
and concluded that the HAVi framework, which originally has been developed 
for home entertainment systems, matches these requirements to a large 
degree. Based on this, we discussed necessary extensions of the HAVi archi- 
tecture. During this discussion, we concentrated on describing the support for 
the MOST bus, which is beginning to become widely used in modern upper- 
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class vehicles. Further extensions have been sketched brieflY and are the focus 
of related work [8], 

By extending the HAVi framework with support for the MOST bus, we also 
showed the possibility of a HAVi that is independent of the underlying com- 
munication bus. The changes necessary for a support of the MOST bus indicate 
the parts of the HAVi architecture that have to be adapted to add support for 
an arbitrary communication mechanism. This would also allow for a HAVi, 
which is for example based on TCP/IP. We believe that such an independence 
from the communication system could also significantly improve the use of the 
HAVi framework in the home entertainment environment. 
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Abstract 

Digital Audio Broadcasting (DAB), based on the Eureka-147 specifi- 
cation is rapidly gaining a foothold in the consumer market, with an 
increasing number of broadcasters in many countries promoting the 
new services that DAB offers, as well as the excellent audio repro- 
duction made possible through digital transmission. 

Additionally, DAB offers significant extra capability for in-car info- 
tainment systems from the large data capability supported by the 
Eureka standard. In the automotive environment this bandwidth can 
be deployed for traffic service information, using standards such as 
RDS-TMC and TPEG. 

Also, emerging applications such as off-board navigation are very 
hungry for bandwidth, and with the right server technology in the 
broadcasting chain DAB offers a very credible alternative to the 
expensive cellular network for delivering large volumes of data to 
individual vehicles. 

As the leading supplier of DAB solutions to the electronics industry. 
Frontier Silicon has also developed automotive DAB products, 
chipsets and modules, that enable 1st tier suppliers to quickly inte- 
grate DAB audio and data capability into their in-car systems. 

This paper will cover the following points: 

► Advantages and applications of DAB in the automotive marketplace. 

► The design challenges inherent in implementing DAB in an automo- 
tive environment. 

► Example automotive DAB solutions. 

► Future roadmap for digital radio. 
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1 Advantages of DAB in the Automotive Environment 

Eureka-147 enjoys a number of significant benefits to the automotive listener, 
and also some significant benefits to the broadcasters. With the DAB service 
being heavily promoted by broadcasters in Europe the combination of these 
benefits and the new content that is being broadcast is making DAB a com- 
pelling feature for cars that will be manufactured in the coming years. 



2 Benefits of DAB 

High quality digital audio with rugged and reliable reception - listeners 
can benefit from a good quality of reception in a moving environment through 
the immunity to interference that is inherent in the DAB system. Coupling this 
with the excellent audio quality that high bit rate digital audio provides 
ensures that listeners soon start to prefer their digital radio over EM. 

Broader range of audio services - as DAB offers more efficient spectrum 
usage than analogue EM a key benefit to the listener is the availability of more 
and diverse services. For example, across the UK there are now over 100 dif- 
ferent radio channels (national and regional) available to the listener. In the 
case of the BBC, DAB has enabled the broadcaster to make available eleven 
high quality audio channels over the DAB coverage area (currently 65% of the 
UK population, rising to 85% in 2004), compared to the four national channels 
available over FM. 

Easy to use and informative receivers - tuning can be easily be performed 
via a text based user interface. The receiver can supply to the listener a com- 
prehensive list of services that are available, a list that can be simply navigat- 
ed via the user interface. This is coupled with the ability of receivers to follow 
a selected service without the need to re-tune. 

More efficient use of broadcast spectrum, through the audio compression 
technologies utilised in DAB broadcasters can achieve significant gains in the 
number of audio services that can be transmitted with good quality in a given 
amount of bandwidth. As has been noted above this feature has already been 
used by broadcasters in the UK to enable them to provide a larger number of 
services than they are able to over FM. 
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3 Data Services - Benefits to Users and Broadcasters 

DAB can be considered as a fast and low cost data pipe. Approximately 20% of 
DAB bandwidth is reserved for data applications, which are split into three 
areas as shown in the table below. 

There is increasing interest in utilising the DAB to connect drivers with more 
useful information such as navigation updates and more detailed traffic infor- 
mation via Ttaffic Message Channel (TMC) and ll’affic Program Expert Group 
(TPEG) standards. These services can be delivered free to air, or using a cellu- 
lar connection to provide a back channel that will enable service providers to 
charge for more valuable services. 



1 Service Name 


Senice Description I 


Sendee Iiifonnation (SI) 

Program Associated Data (PAD) 

Non-program Associated Data (N-PAD) 


Basic RDS like ftmetions to imdeipin tlie 
service 

Used to broadcast information abont tire 
station being received and die content 
being played 

Available for otlier applications such as 
map data, dynamic navigation updates 
etc. 



Table 1 . DAB data applications. 

The data capacity of DAB is increasing in importance with the advent of 
dynamic navigation systems where the quantity of data required to update the 
static maps with traffic news will start to exceed the capacity of the FM RDS 
channel. 



4 Design Challenges 

One of the main characteristics of DAB is the complexity of the air interface. 
This complexity has been driven by the requirement to provide excellent audio 
quality even in poor signal conditions. The signal chain implemented in the 
base band processor requires a great deal of real time computing power, and 
also the interleave depth specified in DAB requires a large quantity of memo- 
ry. 

Coupling these processing requirements with the other environmental chal- 
lenges faced in an automotive environment (temperature, vibration, limited 
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ventilation and extreme cost pressure) makes implementing a cost effective 
automotive DAB system a difficult problem to address. 



5 Processing Trade Offs 

As is noted above, DAB requires a significant amount of processing power. This 
processing power can be provided by a fast general purpose DSP, and exam- 
ples in the market show that a 200 MHz dual MAC DSP can provide sufficient 
horsepower to implement DAB. 

However, it is also worth noting that general purpose DSPs driven at high 
speed also dissipate a lot of power (an example is a DSP that is being put for 
ward as a DAB engine, will dissipate 280 mW at the required 200 MHz), which 
could bring ventilation problems in a crowded modern head unit. 

Because of the power limitations that are inherent in a general purpose DSP, 
another approach is to implement the DAB signal processing chain in dedicat- 
ed hardware. This approach has the benefit of reducing power consumption 
(generally gates will consume less power than a software implementation), but 
this power consumption saving comes at the expense of system flexibility. In 
the DSP approach, spare processing power can be used to implement other 
radio features such as a UI or command interface, whereas in a hardware solu- 
tion this option does not exist and a host processor is required even in the sim- 
plest applications driving up system cost. 

A third approach attempts to marry the DSP and hardware solution to achieve 
a good compromise of flexibility, cost and performance. This approach has 
been taken in the Frontier Silicon Chorus I multimedia processor, and it has 
resulted in a low power (and low cost) DAB base band solution that can also 
perform other tasks such as MP3 decode. 



6 Cost Pressure 

In the consumer market, stand alone DAB receivers are now being sold for 
under €100. Especially in the UK, this has helped the volumes of DAB products 
take off very rapidly; with some estimates showing that over 500 K receivers 
were sold in the UK during 2003. 
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Cost has been driven down by companies like Frontier Silicon deploying high- 
ly integrated base band devices for DAB that significantly reduce the bill of 
materials required to implement a radio, and also providing high performance 
and low cost modules that enable consumer manufacturers to easily imple- 
ment DAB in their products. 

The low component count approach that can now be achieved in a DAB sys- 
tem is demonstrated in the diagram below showing a base band implementa- 
tion for a Frontier Silicon base band module implemented using just three ICs, 
of which two are commodity serial flash and stereo codec devices. 




Fig. 1. Low chip count DAB base band. 



7 Frontier Silicon DAB Solutions 

To implement high performance DAB solutions. Frontier Silicon has teamed up 
with several key players. These included Ensigma (a division of Imagination 
Technologies), TSMC, Flextronics and The Technology Partnership. 

Based on the Frontier Silicon Chorus 1 (FSlOlO) DAB base band, this team has 
developed and produced a range of DAB modules to meet a number of differ- 
ent market requirements. The basic features of these modules are described 
below. 

Chorus I is based on a multi-threaded, dual MAC DSP core operating at 
1 50 MHz, and as it is targeted a DAB applications it includes a number of "DAB 
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accelerators" for computing intensive parts of the DAB system such as the 
Viterbi. To enable fast and low power execution of the various algorithms 
required in a DAB system FSlOlO also includes 384 Kbytes of SRAM. This 
means that the base band can be implemented with just one, low cost, serial 
flash memory for program and parameter storage. 

The DAB hardware and software in the Chorus 1 enable the device to perform 
the demodulation, decoding, protocol stack required to decode DAB using just 
30% of the processing bandwidth available when the device is operating at 
150 MHz, giving a full DAB base band solution that consumes less than 100 W. 



I Module Name 


Target Applicatiotts 


Features I 


Venice (1.1 and 2.0) 
Diablo 


Kitchen Radios, CD boom 
boxes. Micros, Timers, Clock 
Radios 

High End Hi Fi 
All Mains powered audio 
Hand Held/ Pocket Rathos 
Portable radios. Jukeboxes 


Dual Band (Band III and L 
Band) reception, stereo audio 
out. Digital Audio Out LCD 
controllei'. Keypad interface 
Slave DAB module mode 
Dual Band (Band III and L 
Band) reception, Very small 
size, low powei 
consmuption, single powei' 
supply input stereo and 
digital audio out LCD 
conh'oller. Keypad interface 


Roadstei' 


Integrated car indio/CD 
players, naiigation systems, 
telematics 


Dual Band (Band III and L 
Band) reception, small size 
low power consiunption, 
digital audio out automotive 
software extensions 



Table 2. 



8 Block Diagram of Chorus FS1010 Media Processor 

This intelligent combination of hardware and software has resulted in a very 
low power solution that maintains the software flexibility that is characteris- 
tic of DSP based solution. 

As well as a powerful DSP and DAB accelerators. Chorus I is also a very high- 
ly integrated system on a chip (SoC) device. It integrates a large number of the 
peripherals that may be used in a DAB receiver, including a USB interface and 
LCD controller. The aim of this integration is to enable systems to be developed 
with DAB capability, but with maximum flexibility to enable the base band to 
fulfil a number of other system requirements, so reducing manufacturing cost. 
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For example, the Venice and Diablo modules that are based on Chorus 1 are 
capable of performing the user interface function as well as performing DAB 
decode, therefore removing the need for a further microcontroller and associ- 
ated memory and peripherals in the system. 
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I I 10 



Serial I lcd I =dm 

Control I I 
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MMU & External Memory 
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t 



Fig. 2. Block Diagram of chorus FSlOlO media processor. 



9 Roadster Automotive DAB Module 

A real world example of an automotive DAB solution is the Frontier Silicon 
Roadster module, which has been designed in partnership between Frontier 
Silicon and The Technology Partnership. 

The design objectives of Roadster were generated from intensive discussions 
with customers to make the specification as close as possible to the require- 
ments of customers. 

The key requirements that prospective customers put forward were: 

► Good sensitivity and general RF performance. 

► The module should be small enough to fit into a 1 DIN integrated car 
radio/CD player. 
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► Power consumption should be as low as possible. 

► A small number of connections to the main pcb. 

► Controlled as a slave module via a serial interface. 

► Software features such as DAB to FM service linking to support the 
automotive environment. 

► Ability to perform other multimedia tasks such as MP3 decode. 

As the Roadster module will be installed in many different integrated car 
audio product (e.g. a combined DAB + AM/FM Radio with CD) it was decided to 
design the module as a slave device controlled via a serial interface from the 
host microcontroller in the system. This means that it can be easily added to 
an existing product without a significant amount of re-design work. 



10 Cost Optimisation 

The cost pressures in the consumer market are also mirrored in automotive 
systems, so automotive DAB receivers must be designed with cost in mind, 
despite the heavy complexity embedded within them. And the complexity of 
automotive systems continues to increase with increased demand for low cost 
navigation systems, as well as new multimedia functions such as MP3. 

A key opportunity to reduce the cost of implementing DAB in an automotive 
system is through integration. Many 1®^ 2 °^ generation DAB receivers on 

the market are implemented as "black boxes", connected to the audio head 
unit via screened cables and controlled via a CAN interface or via a multime- 
dia bus such as D2B or MOST. 

This approach may be valid where it is not feasible to implement DAB along- 
side the other functions in the audio head unit. However, the "black box" 
approach has added expenses - the module implementing the DAB function 
duplicates several functions in the head unit, including microcontroller/DSP, 
power supplies, network interfaces and connectors. 

A more cost effective approach requires a DAB function that can be imple- 
mented as part of a multi-function audio head unit. This requires that the DAB 
module is small enough (a good benchmark is the size of an existing AM/FM 
tuner) and also does not consume a great deal of power to avoid internal heat- 
ing effects and to minimize the effect on the power supply design. 
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As can be seen from the block diagram, the Roadster module has been 
designed with minimal connections, enabling a simple interface to the rest of 
the system as well as keeping the size of the module to an absolute minimum. 
The dimensions of Roadster are just 55 mm long by 37 mm high by 8mm wide, 
making it competitive in terms of size with most AM/FM tuners. Power con- 
sumption is also very low, with the full receiver consuming just 800 mW while 
decoding a DAB signal and outputting audio. 




(115.2K flash) 

Baud) 

Fig. 3. Roadster block diagram. 

The size and power consumption of Roadster make integration of the DAB 
function into a crowded radio/CD/MP3 IDIN device. 

As well as the standard DAB demodulator, protocol stack and master/slave 
interface that is found in the consumer oriented modules from Frontier Silicon 
the Roadster module also takes advantage of the extra information broadcast 
to support features such as service linking. Improved soft error concealment 
strategies are also included to improve the listening experience in poor cover- 
age areas. 
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11 Example DAB+AM/FM Radio + CD Using Roadster 




Fig. 4. Example DAB + AM/FM radio + CD Using Roadster. 



12 DAB Roadmap 

The demand for DAB receivers is already taking off rapidly, especially in the 
UK, as it is expected to grow rapidly in other European countries as new serv- 
ices become available solely over the digital broadcast medium. In the auto- 
motive market the drive for equipping cars with DAB will come from two 
directions, from car purchasers who want to enjoy the new audio services and 
clearer reception, and from car purchasers who wish to have the latest navi- 
gation technology. 

In both cases, the cost of installing DAB in the car is going to be crtical - it has 
already been noted in this paper that one of the catalysts for DAB in the UK 
has been the availability of receivers costing less the € 100, and this factor will 
also influence car buyers and also the OEMs who make the decision to inte- 
grate the functionality in their new vehicles. 

To meet the increased cost pressure semiconductor manufacturers will have to 
respond with further improvements in their DAB solutions, concentrating on 
reducing the silicon real estate required to implement the DAB function. 








Digital Audio Broadcasting - Solutions for the Automotive Environment I 323 



As DAB matures there is also an increasing trend to use it as a transport medi- 
um for data in other applications. A very real example is the decision of the 
Korean government to introduce a multimedia broadcast service in 2004 called 
DMB (Digital Multimedia Broadcast) that builds on the Eureka 147 specifica- 
tion. 




Fig. 5. Frontier Silicon Roadster on Evaluation Board. 

DAB has taken a long time to arrive, but with the convergence of new audio 
services being broadcast solely over DAB and low cost, consumer friendly 
receivers becoming available it is clear that the time for DAB has come. 
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Abstract 

The purpose of this paper is to show how starting from the customer 
specifications the basic methodologies of software development 
have been applied to the realizations of two ATE (Automatic Testing 
Equipments) for the functional verification and validation of two 
automotive devices with specific complex constraints in the func- 
tional and real time domains, with body/comfort and HMl function- 
alities. 

The problems of formal accuracy and exactness of notation present 
in the software development cycle that may affect the correct func- 
tional and real time behaviour of the resulting target system will be 
analyzed. It is then described which were the general purposes and 
features identified as necessary for the construction of the two 
ATE's, that led to the definition of the hardware and software solu- 
tions. Finally, the technical characteristics of the machines resulting 
from this study are described in two separate paragraphs, were is 
evident how devices from different domain led to different solutions. 



1 Introduction 

The latest generations of automotive electronic devices are introducing more 
and more complex functionalities, with more demands on software and hard- 
ware performances. From the combination of functionalities like voice recog- 
nition, graphic elaboration, and mpeg decodification in the infotelematic 
domain, to the hard real time requirements necessary for safety related sys- 
tems. 

This complexity is obviously reflected in the whole process of the system 
development till its validation. Starting form the customer requirements spec- 
ifications the software life cycle passes through several phases, each one of 
whom can introduce indetermination in the final results. Nowadays this 
process in the automotive applications still suffers of several problems that can 
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influence in a bad way these indeterminations. It's compulsory therefore the 
support of precise methodological foundations. The system validation of the 
device is the ultimate reverse direct link with the initial customer require- 
ments, therefore if performed with a rigorous approach is the consistent con- 
firmation of the effective compliance of the results with the initial design. 
Moreover, the system validation is gaining more importance as the software 
and the hardware are becoming more and more a unique component, and 
their associated performance is becoming sensibly visible and important for 
the final user. 

Due to this increasing complexity the system validation performed with usual 
non-automated tools is time consuming and not adequately efficient. 
Combined functionalities with real time demands specifications require long 
time complex testing activities. Elaborated HMI functionalities are hard to 
measure and validate (e.g. the timing measurements of system response on a 
display panel to different contemporary stimuli coming from the operator and 
field signals). The coverage analysis and insight of the functional domain is evi- 
dently an issue if the system has intricate functions. Furthermore the non- 
automated methods do not allow the correct repeatability of long system "on 
the field" tests. 

This considerations convinced Magneti Marelli's R&D testing department that 
a specific ATE (Automatic Testing Equipment) with a rigorous real-time and 
behavioural engine able to perform complex and repetitive tests would solve 
this problems, improving the efficiency of systems testing and validation. An 
ATE capable of generating test sequences starting from a formal high-level 
description of the system specifications with functional and real time infor- 
mation, and endowed with the appropriate simulators and actuators able to 
fully emulate the operational field of the system was therefore needed. 



2 The Problems in the Software Life Cycle 

During the software development cycle there are many formal accuracy prob- 
lems that we can encounter that affect the correct functional and real time 
behaviour of the resulting target system. Figure 1 shows for reference a com- 
mon V-Model representation of the various stages in the software develop- 
ment. 
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First of all at the present time and for almost every system design, the require- 
ments received from the customer are usually expressed in an informal nota- 
tion, with a large use of natural language, and are comprehensive of function- 
al and non functional behaviour rules, real time aspects, till to the definition of 
EMC and lifetime issues. We usually start therefore from a system's design 
basis that is lacking of a formal description. A direct consequence is a certain 
lack of precision in the description, that is, even if rigorous, quite difficult to 
manage from a technical point of view. 



Fig. 1. The software life cycle. 

Proceeding in the software lifecycle the customer requirements specifications 
are collected, analyzed, discussed and reviewed with the customer itself in 
order to translate them into a consistent software requirements specification. 
This is a first step of conversion that introduces to the problem of "fragmen- 
tation". With "fragmentation" [7] is intended that the link among the different 
phases does not occur through a direct scientific and rigorous conversion but 
may require manipulation and intervention. Therefore the flow of information 
through the different stages is fragmented. Of course this can affect the pre- 
cision of the implementation. 

The software requirements specifications are then the input for the subse- 
quent design development. At the high level design the overall software archi- 
tecture with the relationships among the required software components is 
defined. Real time constraints and performances have to be correctly 
addressed, in consideration of the lower hardware platform. Hardware and 
software partitioning are important decisions at these early stages. In the 
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design procedure, leading from the high level to the low level design, arises 
the problem that not all the software and hardware performances are well 
known. It's impossible to foresee the real time behaviour of software modules 
when executed on the field by the microprocessor, therefore hypothesis on the 
average performance of those modules are used. 

To solve these problems there are many powerful tools that can provide a pseu- 
do-coding starting from a high level design. These software development envi- 
ronments are specifically designed to describe and write real time code that 
runs on embedded systems. In any case, the initial lack of rigorous translation 
of the specifications still exists each time the customers give to the supplier an 
informal requirements description. 

Different studies are carried out on the pursuits of new languages able to 
describe the functional behaviour and timing description in a formal notation 
as, for example, the UML, that recently has reached the version 2.0. Several 
European projects also are studying the problem. Not last, the EAST-EEA 
European project is working to define the ADL, Architectural Description 
Language, a common language for the description of automotive architectures 
and components and the formats to exchange description data between sup- 
pliers and integrators. The ADL is also based on UML standards, and its 
approach separates the automotive electronic architecture in multiple layers, 
from a top vehicle view till to an abstraction description of the lower hardware 
layer. 

Once the software development and testing have a positive result, the follow- 
ing phase, which is the main purpose of this paper, is the realization of the 
System Testing. 



2 Realization of the System Test 

In the following two paragraphs, it is shown how two specific ATE, one for a 
body control device and one for a telematic module, have been first defined 
and then developed, starting form the customer specification requirements. 
The analysis of what the system is and how it operates in its working envi- 
ronment has been the starting point of the work. It must be underlined an 
important characteristic of the system testing: the system test must be per- 
formed with a black box approach, i.e. without opening the system, and run- 
ning the end-user software. The rationale to this approach is that any invasive 
technique used to extract information from the system itself can potentially 
perturbate the measures, and therefore must be avoided. All the interactions 
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have to be performed using the system user interfaces, in the field environ- 
ment or a simulation of it. In other words the system has to remain as it will 
be sold on the market [1]. An intensive analysis on the system's input and out- 
put devices has been therefore carried out to create a perfect reproduction of 
the real stimuli with dedicated simulators (keyboards, audio inputs and out- 
puts, CAN Network, simulation of the GSM network RF signal and GPS RF sig- 
nal, etc.). For all these it has been analysed how to implement on hardware and 
how to pilot their interaction with the system under test. 

Another important issue is the real time domain management. Automotive 
applications have strict real time requirements, which therefore become an 
important feature for the system under test as well for the simulated environ- 
ment to which it is connected (the ATE). 

Beyond the hardware of the ATE there is its core engine that has the purpose 
to manage the testing activities in a real time domain. The main general func- 
tions identified for the ATE are: 

► Simulate the environment around the connections of the device under 
test through an hard real time, event driven control of the simulators. 

► Execute automatically the test sequences that simulate the identified 
necessary operative conditions. 

► Acquire and record all the subsequent conditions/states. 

With the objective to: 

► Verify the application software in the time and value domain. 

► Verify the robustness of the software. 

► Manage the errors. 

► Get evidence of the criticital functionalities. 

ATE's runtime management strategies and programming methodologies have 
been object of studies in collaboration with the technical university 
"Politecnico di Torino" with the intent to obtain an ATE capable of generating 
test sequences starting from a formal high-level description of the system with 
functional and real time information. This would be the way to create the 
strong link with the initial specification that if performed with a rigorous 
approach would represent the consistent confirmation of the effective compli- 
ance of the results with the initial design. The language chosen as standard 
notation is UML. This language is also a valid way to describe timing attrib- 
utes. In fact there's the possibility to describe absolute and relative actions 
timings referred to the sequence diagrams. The input for the ATE would come 
from the same UML high-level description used to describe the scenarios of the 
system behaviour. This would save time and avoid the fragmentation problem 
described before. 
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3 The Realization of the Telematics Domain ATE 

Ixfin Magneti Marelli Electronic Systems designs and produces advanced elec- 
tronic systems and modules for vehicles. The infotelematic system subject of 
the test is a system that delivers to the user services and options like GSM, 
Radio, voice recognition and speech. Audio CD, WAP, MP3 and Navigation with 
GPS. A very important characteristic of any infotelematic system is the HMI 
(Human Machine Interface), which allows the user to interact with the system 
trough the video and a set of keys and/or knobs. 




Fig. 2. Telematic ATE. 



The software verification is a part of the development process that is very 
important on a system with this complexity. With these level of HW and SW a 
lot of functions are working at the same time (multitasking environment) and 
it is very difficult to simulate the environment interaction, i.e. the operations 
performed by the user, the car network, GPS end GSM signals, etc. 
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The solution: 

Ixfin Magneti Marelli R&D Testing department has developed an Automatic 
Test Equipment (ATE) based on a "National Instrument" PXl computer instru- 
mentation with vision, motion, speech synthesizer, actuators and special test 
sets, controlled by LabView and test stand environment. More in details the 
test equipment is based on two parts: 

► The main PXI System, including: the CAN interface, the audio card for 
the speech synthesizer and the voice recognition; an image acquisition 
board; a digital I/O board used to control the internal matrix of the main 
PXI System; a multifunctional I/O device used to generate or acquire 
digital signals; a controller of the instrumentation on the main PXI. The 
other instrumentation used is the GSM network simulator, the radio 
broadcasting synthesizer simulator and the RDS generation. As a GPS 
system a dedicated device directly connected on the specific connector 
of the infotainment module and able to simulate the RF satellite signal, 
that would be received during some customizable trips, is used. 

► The slave PXI, more related to the realization of the HMI interaction 
which consists in an automatic mechanical actuator for the keys press- 
ing, controlled trough a PEG managed by a PC serial line, a colour cam- 
era for the verification of the Display, and two devices for the automat- 
ic insertion/ejection of the CD and of the SIM card for the telephone. 

With this structure it is possible to perform robustness tests simulating the use 
of the product in the car, running at the same time sequences of emulation of 
interaction with the user (HMI), during parallel processing of radio signal, 
RDS, GSM communication, GPS reception, CD-Player, data management, and 
communication with the car trough the CAN interface. 



4 The Realization of the Body Domain ATE 

This has been the realization of an ATE to test devices for body/comfort appli- 
cations, i.e., internal and external lights, commands, serial interfaces, compre- 
hending 2 CAN network interfacing with gateway functionalities. This system 
required more rela time constraints that influenced the choice of the operative 
system running on the ATE. First the customer specifications of the device 
have been analysed. The analysis has produced the specifications for the sys- 
tem ATE, divided in hardware and software. In the hardware specifications 
have been described the requested features: 

► Power management of the device under test. 

► Analogical, logic and frequency signal generation. 
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► Fault simulation on every contact of the device, i.e. short circuit to 
ground and to battery, and disconnection. 

► Electrical characteristics of the device under test as well for the expect- 
ed loads connected. 

The software specifications required: 

► The possibility to create and modify test sequences. 

► The possibility to graphically monitor the stimuli produced by the ATE 
and the output generated by the system. 

► The possibility to manage and configure all the CAN network messages. 

► Generated with timing relations or event based. 

► Reporting. 




Fig. 3. Body ATE. 



The Solution: 

The hardware of the system obtained is composed of a PXl rack with the 
boards dedicated to controller, serial communication ports (CAN, RS232), sig- 
nal matrices, power matrices, resistor matrices, I/O to GND/Vbat TTL condi- 
tioners, relays, characteristic impedances, etc. The electrical connection of the 
device under test is obtciined with a specific fixture. 
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The software is composed of four combined tools that allow the following test- 
ing procedure: to generate on a host PC a test pattern (pattern editor), down- 
load it on the real-time controller of ATE (RT controller manager), execute it 
with the report generation of the events (real time program), download it on 
the Host PC (RT controller manager), and analyze it to check the behaviour of 
the device (data analyzer). In particular: the pattern editor allows and assists 
the user in the definition of a testing sequence, using functions databases and 
mnemonic names for the system resources; the data analyzer allows to analyze 
the values of the events and their correlations in the time domain, also show- 
ing the data passed on the communications networks. 



5 Conclusions 

With the use of the testing equipment and methodologies described above, 
Ixfin Magneti Marelli Electronic Systems gained many advantages in terms of 
reliability, time-to-market, and development costs. Our software robustness 
tests assure that the product in output is more reliable; the time for verifica- 
tion is reduced and it's not necessary to dispose of a real prototype car for the 
testing. Complex, repeatable, specifically aimed tests that a normal human 
user could not perform, with this equipment can be. This helped for example 
to find out anomalies and problems that were occurring only after several 
cycles, and that a manual testing could not point out. Moreover through the 
automated testing it has become easier to guarantee the functional reliability 
and no regression capability on previously developed software parts. 

From the methodological point of view, is obvious that is becoming more and 
more necessary a standardization of the languages and the formats for 
exchange of description data between suppliers and integrators that would 
definitively set up the strong rigorous approach for software developments. 
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Abstract 

Digital convergence, in recent history, has been prevalent in the con- 
sumer equipment domain and the design engineers in this area have 
been struggling with a plethora of emerging standards and proto- 
cols. What lessons can we learn from their struggle? The same 
dilemmas now exist in in-vehicle telematics and infotainment sys- 
tems but with the added issues of extremes of temperature, safety, 
security and time in market. 

This paper will first describe and position each emerging in-vehicle 
standard and their respective strengths and weaknesses. It will then 
explore the designers dilemma: how to build flexible and scalable 
system architectures which will allow the time in market of telem- 
atics platforms match that of the host vehicle whilst still communi- 
cating internally and to other external systems. It will then go on to 
discuss enabling technologies and how to implement re configurable 
and upgradeable telematics platforms that can cater for protocols 
today and in the future. 



1 Introduction 

In-car electronics content has seen tremendous growth in recent years, not 
only in the traditional body control and engine management areas but in the 
new areas of driver assistance systems and telematics type applications. 
Figures recently published by the IEEE show rises in car electronics of up to 
16% and they announced that by 2005 electronics will account for 25% of the 
cost of a mid-size car. 

One of the high growth area is telematics systems (the convergence of mobile 
telecommunications and information processing in cars) which exhibit charac- 
teristics more like those of consumer products i.e. short time to market, short 
time in market and changing standards and protocols. These characteristics 
are the complete opposite to those of traditional in-car electronics, which could 
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potentially cause issues from design right through to manufacture and servic- 
ing. These issues have an impact on the way designers are approaching 
designs and also which in-car bussing system to choose. 

The traditional in-car bus systems based on a serial, event triggered protocol 
such as CAN and J1850 have been used in the body control area successfully 
for many years but bandwidth and speed restrictions will make it difficult for 
these to be used in the newer real time applications. A range of new bus stan- 
dards have been emerging such as time triggered protocols and optical data 
busses to meet these new data through put challenges. In-car bus networks 
can be divided into four main categories: 

► Body Control - covering data and control signals between car seat con- 
trols, dashboard/instrument panel clusters, mirrors, seat belts, door lock 
and airbags (passive safety). 

► Entertciinment and Driver Information Systems - communication and 
control between radio, web browser, CD/DVD player, telematics and 
infotainment systems. 

► Under the hood - networking between ABS brakes, emission control, 
power trmn and transmission. 

► Advanced Safety Systems - data transfer between brake-by-wire, steer- 
by-wire and driver assistance systems (active safety). 

We will look at each area in turn and discuss the different emerging vehicle 
emerging standards and protocols that apply to each area. Figure 1 and 
figure 2 show the various in-car networks and their intended applications. 



Under the 
Hood 



Body 

electronics 



In Cockpit 

Information Entertainment 



CAN 



-Safe-by-wire— 
• LIN • 



IDB-C 



• D2B — 

IEEE-1394 

. MOST - 

• IDB-1394 

GMLAN 



Fig. 1. Applications for in-vehicle networking. 
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2 Body Control and ‘Under-the-Hood’ Busses 

2.1 Controller Area Network (CAN) 

In the mid-1980s Bosch developed the Controller Area Network (CAN), one of 
the first and most enduring automotive control networks. CAN is currently the 
most widely used vehicle network with more than 100 million CAN nodes 
installed. A typical vehicle can contain two or three separate CANs operating 
at different transmission rates. A low speed CAN running at less than 
125 Kbps for managing the body control electronics such as seat and window 
movement controls and other simple user interfaces. Low speed CANs have an 
energy saving sleep mode in which nodes stop their oscillators until a CAN 
message wakes them. Sleep mode prevents the battery from running down 
when the ignition is turned off. 



Active safety 

Passive Safety X-by-wire (e.g. driver 
Electronics assistance) 



• ByteFlight (10 Mbits/s) • 

FlexRay (lOMbits/s) • 

• HP (25 Mbits/s) • 

• TTCAN (1 Mbit/s) • 

Fig. 2. Safety Critical Busses. 

A higher speed (up to 1 Mbps) CAN runs more real-time-critical functions such 
as engine management, antilock brakes and cruise control. Controller-area net- 
work (CAN) protocols are becoming standard for under-the-hood connectivity 
in cars, trucks and off-road vehicles. One of the outstanding features of the 
CAN protocol is its high transmission reliability, which makes it well suited for 
this type of application. The CAN physical layer is required for connections to 
under-the-hood devices and in-car devices.CAN has found its way into general 
industrial control applications such as building management systems and lift 
controllers but more surprisingly is now being considered for use in telecoms 
type applications such as Base Ifansceiver Stations (BTS) and Mobile Switching 
Centers (MSC) equipment. This is due to CANs protocol error management, 
fault isolation and fault tolerance capabilities. 
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2.2 Local Interconnect Network (LIN) 

The Local Interconnect Network (LIN) has been developed to supplement CAN 
in applications where cost is critical and data transfer requirements are low. 
Conceived in 1998, the LIN consortium comprises car manufacturers Audi, 
BMW, DaimlerChrysIer, Volvo and VW. LIN is an inexpensive serial bus used 
for distributed body control electronic systems in vehicles. It enables effective 
communication for smart sensors and actuators where the bandwidth and ver- 
satility of CAN is not required. Typical applications are door control (window 
lift, lock and mirror control), seats, climate regulation, lighting and rain sen- 
sors. Outside the automotive sector LIN is used for machine control as a sub- 
bus fro CAN. 

LIN is a UART-based, single-master, multiple-slave networking architecture 
originally developed for automotive sensor and actuator networking applica- 
tions. The LIN master node connects the LIN network with higher-level net- 
works, like CAN, extending the benefits of networking all the way to the indi- 
vidual sensors and actuators. 



3 Entertainment and Driver Information Systems 

Car infotainment and telematics devices, especially car navigation systems, 
require highly functional operating systems and connectivity. Until now the 
existence of both open-standard and proprietary stand-alone buses has worked 
quite well. But because of their integrated nature, future systems will require 
electronic subsystems to work together. 

By relying on open industry standards, all key players - from manufacturers 
to service centers and retailers - can focus on delivering core expertise to the 
end user, rather than expending the time and effort it would take to develop 
separate, incompatible designs for specific vehicles or proprietary computing 
platforms. Future systems will be highly integrated, open and configurable. 

Several organizations and consortia are leading the standardization efforts, 
including the MOST Cooperation, the IDB Forum and the Bluetooth Special 
Interest Group (SIG). 
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3.1 Media Oriented System Transport (MOST) 

MOST networks, managed by the MOST Cooperation, are used for connecting 
multiple devices in the car including car navigation, digital radios, displays, 
cellular phones and DVDs. MOST technology is optimised for use with plastic 
optical fibre, supports data rates of up to 24.8 MBit/s, up to 24 nodes, infinite 
length, is highly reliable and scalable at the device level, and offers full sup- 
port for real-time audio and compressed video. MOST technologies are strong- 
ly supported by German automakers and suppliers. The MOST bus is endorsed 
by BMW, Daimler Chrysler, Harman/Becker and OASIS Silicon Systems. A 
recent notable example of MOST implementation was its use by Harman 
Becker in the latest BMW 7 series. Optical data buses in cars have recently 
been favoured for their noise immunity characteristics but there is concern 
that repeated connection and re-connection of the optical connectors has been 
found to cause damage to the optical fibre (typically 5-6 re-connects). Optical 
fibre fractures and damage is notoriously difficult to find so some automotive 
companies are reviewing MOST with a view to maybe moving back to copper 
wire instead of optical fibre. 



3.2 Intelligent Transport System Data Bus (IDB) 1394 and CAN 

The IDB Forum manages the IDB-C and lDB-1394 buses and standard IDB 
interfaces for OEMs for the development of after-market and portable devices. 
Based on the CAN bus, IDB-C is geared toward devices with data rates of 
250 kbits/s with up to 16 nodes. Applications for IDB-C include connectivity 
through consumer devices such as digital phones, PDAs and audio systems. 

lDB-1394 (based on IEEE-1394 Firewire) is designed for high-speed multime- 
dia applications that require large amounts of information to be moved quick- 
ly on a vehicle. lDB-1394 is a 400 MBaud network using fibre optic technolo- 
gy with a maximum of 16 nodes over 72 meters. Such applications include 
DVD and CD changers, displays, and audio and video systems. lDB-1394 also 
allows 1394 portable consumer electronic devices to connect and interoperate 
with an in-vehicle network. Zayante is one supplier who currently has 194 
physical layer devices for the consumer market and demonstrated IEEE- 1394 
with the Ford Motor Company. The demonstration included plug and play 
connections of a digital video camera, Sony Playstation II as well as two video 
displays and a DVD player. 

Already a leading standard in consumer electronics, 1394 is now almost ready 
for prime time in vehicles. It's ability to simultaneously stream multiple chan- 
nel of audio and high-quality video make it the technology that can transform 
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vehicles into the multi-screened rolling entertainment centres envisioned in 
many concept cars. And work is underway to make it practical for the 
demanding automotive environment. 

Some still think that 1394 is "over kill" for the automobile whereas others view 
it as the next stage after MOST. There is also a view that MOST and 1394 will 
coexist, MOST for the embedded infotainment applications and 1394 for high 
speed, real time video applications. The interest for 1394 is growing in Japan 
and Europe but North American manufacturers still appear to be neutral. 

1394 should start showing up in production vehicles in 2006, but may appear 
in after market products sooner. Renault are publicly leading the charge for 
1394 adoption. 



3.3 The Digital Data Bus (D2B) Communications Network 

The Digital Data Bus (D2B) is a networking protocol for multimedia data com- 
munication integrating digital audio, video and other high data rate synchro- 
nous or asynchronous signals, can run up to 11.2 Mbps, and can have up to 50 
nodes (copper) and 24 nodes (optical). The data bus can be built around either 
unshielded twisted pair (UTP), named 'SMARTwire' or a single optical fibre. 
The maximum bus length for copper is 1 50 meters and no limit to the optical 
variant. This communication network is being driven by C&C Electronics in 
the UK and has seen industry acceptance from Jaguar and Mercedes Benz. For 
example the Integrated Multimedia Communication System that has been 
deployed in the Jaguar X-lVpe, S-lVpe and new XJ saloon. 

D2B optical is self configuring on start up, adapting to what ever devices are 
present on the network at the time. This means that new devices can easily be 
fitted to the network at any time during its life. Car makers that use the D2B 
optical multimedia system will find the standard evolves in line with new tech- 
nologies as they are introduced. The standard will be backwards compatible 
ensuring that new products can be added to a cars system during its life time. 
D2B optical is based on an open architecture which simplifies expansion as no 
changes to the cable harness are required when adding a new device or func- 
tion to the optical ring. The buss uses just one cable, either a polymer optical 
fibre or copper to handle the in-car multimedia data and control information. 
This gives better reliability, fewer external components and connectors and a 
significant reduction in overall system weight. 
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3.4 Bluetooth 

Bluetooth wireless technology is a low-cost, low-power, short-range radio tech- 
nology for mohile devices and for WAN/LAN access points. It is a computing 
and telecommunications industry specification that descrihes how mobile 
phones, computers and PDAs can easily interconnect with each other, and 
with home and business phones, and computers using a short-range wireless 
connection. Bluetooth is an open wireless standard that uses low-power, short- 
range (about 35 ft.) radio signals at the unlicensed 2.4 GHz frequency. It has a 
spread-spectrum, full-duplex signal that hops frequencies at up to 
1600 times/sec to reduce interference, and a data transfer rate of about 
1 Mbps. 

A driver will be able to use a Bluetooth cordless headset to communicate with 
a cellular phone in his or her pocket, for example. As a result, driver distrac- 
tion can be reduced and safety increased. The automotive industry has creat- 
ed a special-interest group (SIG) for the definition of Bluetooth car profiles. The 
SIG includes such members as AMIC, BMW, DaimlerChrysler, Ford, GM, 
Toyota and VW. One example of Bluetooth deployment in cars is Johnson 
Controls BlueConnect™ which is a hands-free system that allows drivers to 
keep their hands on the wheel while staying connected through a Bluetooth- 
enabled cellular phone. 

There has however been some concern voiced over the long term support of 
Bluetooth devices and how noisy in-car environment will effect its operation. 
The lifecycle of cars and other vehicles is very much longer than that for con- 
sumer products or mobile phones so this mismatch between support and serv- 
ice timescales must be addressed by silicon manufacturers. However the 
Chrysler Group showed the use of Bluetooth connectivity in its vehicles at 
Convergence 2002. 

Adoption of Bluetooth has been steadily increasing since its OEM automotive 
debut in the 2003 Saab 9-3. Although Bluetooth will continue its momentum 
in new 2004 models from Acura, Audi, Lexus, Lincoln, Maybach, and Toyota, 
to name a few, its role will be limited, according to the findings of a new 
research study from ABI. 

Protocols based on 802.11, such as DSRC (Dedicated Short Range 
Communications), promise to fill the need for longer range, higher bandwidth 
applications that will not only link vehicles with roadside data access points, 
but between each other. Ratification of the new DSRC protocol is expected by 
the end of 2004, with aftermarket automotive offerings quickly following suit. 
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Aftermarket 802.11-based automotive products will appear by the end of 
2003, initially focused on entertainment applications. 

While the first wave of Bluetooth devices in the vehicle will centre around 
telephony, newer applications will soon follow. These include remote vehicle 
diagnostics, lower-cost telematics services, advanced automotive safety sys- 
tems, vehicle-to-vehicle communications, and remote audio and video down- 
loads into the vehicle, among others. 

Its still remains to be seen if in traffic jams with multiple users using their cell 
phone Bluetooth hands free kits whether performance and audio degradation 
will be an issue. 



3.5 Ethernet 

Ethernet is gaining popularity in the industrial automation segment due to it 
being tried and tested, easy to install and maintain and less expensive than 
other alternatives. When first adopted in the mid-1980s as the Institute of 
Electrical and Electronics Engineers (IEEE) 802.3 standard, Ethernet was con- 
sidered unsuitable on the plant floor. The network protocol was (and still is) 
nondeterministic. Device response times could not be guaranteed because of 
data collisions and the delays in retransmitting data. Data throughput was 
slow. The network medium was subject to electromagnetic interference (EMI). 
Ethernet now runs over shielded and unshielded twisted pair copper, coaxial, 
and fully EMI-resistant fibre-optic cable. Ethernet operating speeds across 
even conventional wire cabling have increased at least an order of magnitude, 
from 10 Mbps to 100 Mbps. Automatic switches that negotiate 10/100 Mbps 
are commonplace, thereby optimizing speed versus service, as well as letting 
you mix 10 Mbps (Ethernet) and 100 Mbps (Fast Ethernet) devices on the same 
network. 



3.6 Universal Serial Bus (USB) 

Universal Serial Bus (USB) has in the past been favored by aftermarket prod- 
uct suppliers such a Clarion (AutoPC). The transmission media is shielded 
twisted pair and runs up to 12 Mbps with up to 127 nodes. The future of USB 
use in the vehicle is uncertain as it may follow the consumer trend and be sup- 
planted by Firewire. It must however be noted that USB can provide up to 127 
whereas Firewire only provides for 16 but perhaps over 16 nodes in a vehicle 
for infotainment may never be exceeded. 
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4 Advanced Safety Systems 

4.1 FlexRay 

FlexRay is a new network communication system targeted specifically at the 
next generation of automotive applications or "by- wire" applications. The by- 
wire applications demand high-speed bus systems that are deterministic, fault- 
tolerant, and capable of supporting distributed control systems. BMW, Daimler 
Chrysler, Philips Semiconductors, Motorola, and the newest member, Bosch, 
are working together to develop and establish FlexRay as the standard for next 
generation applications. 

The FlexRay Communication System is more than a communications protocol. 
It also encompasses a specifically designed high-speed transceiver, and the 
definition of hardware and software interfaces between various components of 
a FlexRay 'node'. The FlexRay protocol defines the format and function of the 
communication process within a networked automotive system. 

The technology is designed to meet key automotive requirements like depend- 
ability, availability, flexibility, and a high data rate to complement the major in- 
vehicle networking standards - CAN, LIN, and MOST. 

With the increasing amount of data communication between the vehicle's 
electronic control units (ECUs), it is important that a high data rate can be 
achieved. FlexRay is initially targeted for a data rate of approximately 
10 Mbit/s; however, the design of the protocol allows much higher data rates 
to be achieved. 

FlexRay, as a scalable communication system, allows synchronous and asyn- 
chronous data transmission. Depending on the application needs, the commu- 
nication cycle can be purely synchronous, purely asynchronous, or a mixture 
of both. The synchronous data transmission enables time triggered communi- 
cation to meet the requirement of dependable systems. The asynchronous 
transmission, based on the fundamentals of the byteflight™ protocol, allows 
each node to use the full bandwidth for event driven communications. 

FlexRay's synchronous data transmission is deterministic with a minimum 
message latency and message jitter guaranteed. FlexRay supports redundan- 
cy and fault-tolerant distributed clock synchronization for a global time base, 
thus keeping the schedule of all network nodes within a tight, predefined pre- 
cision window. 
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4.2 Time Triggered Protocol (TTP) 

Designed for real-time distributed systems that are hard and fault tolerant, the 
time triggered protocol (TTP) ensures that there is no single point of failure. 

TTP is the mature network solution that is low cost and can handle safety-crit- 
ical applications. TTP silicon support has been available since 1998. Second 
generation silicon, supporting communication speeds of up to 25 Mbit/s, is 
available today. TTP has been adopted by aerospace companies due to its rig- 
orous and plain safety approach. TTA Group, which is a governing body for 
time triggered protocol, has a membership including Audi, PSA, Renault, NEC, 
TTChip, Delphi & Visteon. 
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Fig. 3. In-car network speeds. 

4.3 Time Triggered Can (TTCAN) 

The communication in the classic CAN network is event triggered; peak loads 
may occur when the transmission of several messages is requested at the 
same time. CAN's non-destructive arbitration mechanism guarantees the 
sequential transmission of all messages according to their identifier priority. 
For real-time systems, a scheduling analysis of the whole system has to be 
done to ensure that all transmission deadlines are met even at peak bus loads. 
To overcome this potential issue TTCAN was conceived. 
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TTCAN is an extension of the CAN protocol and has a session layer on top of 
the existing data link layer and physical layers. The protocol implements a 
hybrid, time triggered, TDMA schedule, which also accommodates event-trig- 
gered communications. TTCANs intended uses include engine management 
systems and transmission and chassis controls with scope for x-by-wire appli- 
cations. 



5 FPGAs Alleviate the Design Dilemma 

The next few years will be a proverbial mine field for automotive electronics 
designers and choosing the correct data bus will be crucial to the success of 
integrating and testing units in production and long after the car has rolled off 
the assembly line. For Tier 1 suppliers and aftermarket unit design companies 
the problem is amplified as they will potentially supply units to many OEMs 
who will invariably all opt for different data busses and protocols. The indus- 
try has seen a huge shift of design philosophy away from designing a differ- 
ent unit for every OEM and indeed every car model to re configurable plat- 
forms. Reconfigurable platforms cleverly partitioned between software and 
reprogrammable hardware will allow the designer to change the choice of sys- 
tem bus or interface late in the design process and even in production. The 
reconfigurable system concept also enables different standards and protocols 
to be tried, tested and put on road trials and if you they are not found to be 
suitable another bus interface can be loaded into the system and tried out until 
the best configuration is found. 

Whilst this may seem like a designers nirvana and unobtainable it can be real- 
ized today by utilizing programmable logic devices (PLDs) in the form of Field 
Programmable Gate Arrays (FPGA's) and Complex Programmable Logic 
Devices (CPLDs). PLDs can hand back to the designer the control over all phas- 
es of design from prototype, through pre-production and all phases of produc- 
tion. This flexibility and control can be lost when developing systems based on 
ASIC's and ASSP devices. PLD's can also alleviate over stocking and inventory 
issues as these generic devices can be used across many projects and are not 
application specific like ASICs. Once the programmable logic based unit is on 
the road it can even be reconfigured remotely via a wireless communication 
link to allow for system upgrades or extra functions. 

The reconfigurable hardware platform can be brought to market quickly by 
utilizing drop in IP core blocks. For example Memec Design recently announ- 
ced the availability of a cost optimised CAN core interface. The Memec CAN 
core contains the complete data link layer, including the framer, transmit and 
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receive control, error core design and flexible interface enables access to each 
internal status and frame reference. Bit rate and sub-bit segments can be con- 
figured to meet the required timing specification of the connected CAN bus. 
Error counters and error interrupt events report errors. The core is designed 
to provide a bus bit rate of up to 1 Mbps, with a minimum core clock frequen- 
cy of 8 MHz. The CAN core can provide an interface between the message fil- 
ter, the message priority mechanism and various system functions such as 
sensor/activator control. Alternatively, it can he embedded into a system appli- 
cation interfacing with the microprocessor and various peripheral functions. 
Another example is the LIN core available from Intelliga. The LIN core, iLIN, 
is supplied as a reference design that uses a synchronous 8-hit general purpose 
micro-controller interface with minimal buffering for the transportation of 
message data. In addition, the reference design includes a single slave mes- 
sage response filter and a software interface that allows the connected micro- 
controller to perform address filtration. 
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Fig. 4. In-car multimedia design showing functions embedded in an 
FPGA. 

IP cores can he used as part of a more complex design to provide the interface 
to the CAN or LIN hus instead of using a discrete device. Reducing component 
count has many benefits including lowering overall system cost, reducing 
inventory, increasing system reliability and also can help reduce PCB com- 
plexity and layers. Figure 4 shows a generic in-car multimedia design showing 
the use of the CAN core coupled with PCMCIA interface, PCI bridging, IDE 
interface plus other functions. These functions can be modified, changed or 
enhanced during the design phase or modified depending on the end customer 
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requirements. In this way one PCB can be used for many customers with cus- 
tomisation taking place within the FPGA instead of at the board level. This 
model can be extended to include modification or upgrades in the field utilis- 
ing a wireless connection to re-configure the FPGA in system. 



6 Conclusion 

It is clear that there are numerous emerging in-car bus systems providing data 
and control signals to and from nodes as diverse as door locks to highly sophis- 
ticated multimedia terminals. The needs of each bus are different, at the low 
end we need low cost relatively low speed busses and at the high end we 
require high speed, real time data transfer over optical media. Many automo- 
tive OEMs are backing more than one standard due to uncertainties over 
which one will eventually prevail. As a designer these uncertainties can delay 
development and ultimately lead to lost revenue. A solution designers are 
turning to are reconfigurable systems based on FPGAs that can be repro- 
grammed to accommodate changing standards and protocols late in the design 
process and even in production. The bus interface can take the form of pre-ver- 
ified drop in IP Cores that can safe time and effort and thus increase time to 
market. 
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